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ABSTRACT 



The National Assessment of Educational Progress (NAEP) is 
the nation's only ongoing survey of what students know and can do in various 
academic subject areas. This report summarizes the results of the 1998 NAEP 
civics assessment for the nation. The results are based on an assessment of 
students at grade 4, grade 8, and grade 12. Students' performance is 
described in terms of their average civics score on a 0 to 300 scale and in 
terms of the percentage of students attaining each of three achievement 
levels: (1) basic; (2) proficient; and (3) advanced. The achievement levels 

are performance standards adopted by the National Assessment Governing Board 
(NAGB) as part of its statutory responsibilities. In addition to providing 
average scores and achievement level performance for the nation, this report 
also provides results for siibgroups of students defined by various backgroxmd 
and contextual characteristics. Further, the report explores relationships 
between selected teacher and classroom activities and student performance. 

The chapter titles include: (1) "Average Scale Score and Achievement Level 

Results for the Nation"; (2) "Average Civics Scale Score Results for Selected 
Subgroups"; (3) "Civics Achievement Level Results for Selected Subgroups"; 

(4) "Contexts for Learning Civics: School /Teacher Policies and Practices"; 
and (5) "Contexts for Learning Civics: Classroom Practices and Student 
Variables . " The appendices include an overview of procedures used in the 
assessment, standard errors, and a list of the members of the NAEP civics 
development and standing committees . (LB) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



iAfioNWL Center’ itoft EDUcAttoN statistics 






what is The Nation's Report Card? 

THE NATION’S REPORT CARD, the National Assessment of Educational Progress (NAEP), is the only nationally 
representative and continuing assessment of what America’s students know and can do in various subject areas. Since 1969, 
assessments have been conducted periodically in reading, mathematics, science, writing, history, geography, and other 
fields. By making objective information on student performance available to policymakers at the national, state, and local 
levels, NAEP is an integral part of our nation’s evaluation of the condition and progress of education. Only information 
related to academic achievement is collected under this program. NAEP guarantees the privacy of individual students and 
their families. 

NAEP is a congressionally mandated project of the National Center for Education Statistics, the U.S. Department of 
Education. The Commissioner of Education Statistics is responsible, by law, for carrying out the NAEP project through 
competitive awards to qualified organizations. NAEP reports directly to the Commissioner, who is also responsible 
for providing continuing reviews, including validation studies and solicitation of public comment, on NAEP’s conduct 
and usefulness. 

In 1988, Congress established the National Assessment Governing Board (NAGB) to formulate policy guidelines for 
NAEP. The Board is responsible for selecting the subject areas to be assessed from among those included in the National 
Education Goals; for setting appropriate student performance levels; for developing assessment objectives and test 
specifications through a national consensus approach; for designing the assessment methodology; for developing guidelines 
for reporting and disseminating NAEP results; for developing standards and procedures for interstate, regional, and national 
comparisons; for determining the appropriateness of test items and ensuring they are free from bias; and for taking actions 
to improve the form and use of the National Assessment. 
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The National Assessment of Educational Progress (NAEP) is the nation’s only 
ongoing survey of what students know and can do in various academic subject 
areas. Authorized by Congress and administered by the National Center for 
Education Statistics in the U.S. Department of Education, NAEP regularly 
reports to the public on the educational progress of students in grades 4, 8, 
and 12. In 1998 NAEP conducted a national assessment of civics knowledge 
of students in each of these grades. 

This report presents the results of the NAEP 1998 civics assessment for the 
nation. The results are based on assessing a sample of students at each grade that 
is statistically representative of the entire nation. In total, 5,948 fourth graders, 
8,212 eighth graders, and 7,763 twelfth graders were assessed. For this subject 
assessment, in contrast to the other major subject reports presented this year in 
reading and writing, there were no additional state-level results. Students’ 
performance on the national assessment is described in terms of their average 
civics score on a 0 to 300 scale and in terms of the percentage of students 
attaining each of three achievement levels: Basicj Proficient^ and Advanced. 

The achievement levels are performance standards adopted by the National 
Assessment Governing Board (NAGB) as part of its statutory responsibilities. 
The achievement levels are collective judgments by broadly representative panels 
of classroom teachers, education specialists, and members of the general pubhc. 

As provided by law, the Acting Commissioner of Education Statistics, upon 
review of a congressionally mandated evaluation of NAEP, has determined that 
the achievement levels are to be considered developmental. However, both the 
Acting Commissioner and NAGB beheve these performance standards are usefiil 
for understanding student achievement. They have been widely used by national 
and state officials, including the National Education Goals Panel, as a common 
yardstick of academic performance. 

In addition to providing average scores and achievement level performance 
for the nation, this report also provides results for subgroups of students defined 
by various background and contextual characteristics. Further, the report 
explores relationships between selected teacher and classroom activities and 
student performance. A summary of the major findings fi*om the NAEP 1998 
civics assessment is presented on the following pages. 
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Civics Scale Score and Achievement Level Results 



Overall results 

► Average scale scores for the nation were set at 150 on a scale of 0 to 300 for 
all grades assessed (4, 8, and 12). This average can be used as a common 
reference point within grades and for comparisons among population 
subgroups. 

► At grades 4, 8, and 12, the percentages of students performing at or 
above the Basic level of civics achievement were 69, 70, and 65 percent 
respectively; the percentages performing at or above the Proficient level 
were 23, 22, and 26 percent respectively. Two percent of the students at 
grades 4 and 8, and 4 percent at grade 12 performed at the highest 
achievernent level. Advanced. 

Results for student subgroups 

Gender 

► Female students had higher average scale scores than male students at 
grades 8 and 12, but not at grade 4. At grades 8 and 12, the percentages of 
students at or above the Basic level were higher among females than among 
males. At grade 12, however, the percentage of students at the Advanced 
level was higher among males than females. 

Race/Ethnicity 

► At grade 4, White students had higher average scale scores than their 
Asian/Pacific Islander peers, who in turn had higher average scale scores 
than their Black, Hispanic, and American Indian counterparts. At this grade, 
Hispanic students scored at lower average levels than members of other 
ethnic groups. At grades 8 and 12, White students had higher average scale 
scores than Black, Hispanic, and American Indian students. At grade 8, 
Black and Asian/Pacific Islander students scored at higher levels than 

their Hispanic counterparts. At grade 12, Asian/Pacific Islander students 
perforrned at a higher level than their Black, Hispanic, and American 
Indim peers. 

► At all three grades, the percentages of White students at or above the 
Proficient achievement level were higher than those of Black students, 
Hispanic students, and American Indian students. A higher percentage of 
White students than Black or Hispanic students reached the Advanced level 
at grade 12. At grade 8, a higher percentage of Asian/Pacific Islander 
students than Black, Hispanic, or American Indian students were at or 
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above the Proficient level. A higher percentage of Asian/Pacific Islander 
students also reached the Proficient level at grade 4 than did their Black 
and Hispanic counterparts. 

Parents' level of education 

► Consistent with findings of past NAEP assessments, higher levels of parental 
education were associated with higher levels of student performance, 
especially at the upper grades. For example, as high school seniors reported 
higher levels of parental education, their average scores increased. 

► A larger percentage of grade 4 students who reported that their parents 
graduated from college than students who indicated their parents did not 
graduate from high school were at or above the Proficient 2 ichic\cmcnt level. 
At grade 8, higher percentages of students whose parents were reported 

to have graduated from college were at the Advanced level, or at or above 
the Proficient level, than were students from the other parental education 
groups. The percentage of grade 8 students at the Basic level was highest 
among students whose parents graduated from college and lowest among 
students whose parents did not graduate from high school. At grade 12, 
groups with higher parental education levels had higher percentages of 
students at or above each of the achievement levels. 

Region of the country 

► Students in the Northeast and Central regions had higher average scale 
scores than those in the Southeast and West at grades 4 and 8. In addition, 
at grade 4, students in the Southeast outperformed those in the West. 

At grade 12, students in the Central region outperformed their peers 
elsewhere, while seniors in the Northeast had higher average scores than 
those in the Southeast. 

► At grade 4, the percentage of students at the Advanced level was higher in 
the Central region than in the West. At all three grades, higher percentages 
of students in the Northeast and Central regions were at or above the 
Proficient level than in the Southeast. At grades 4 and 8, the Northeast and 
Central regions had higher percentages of students at or above Basic than 
did the Southeast and the West. At grade 12, the percentage of students 

at or above Basic was higher in the Central region than in the Southeast 
and West. 
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Type of location 

► At grades 4 and 8, students from schools in urban fringe/large town and 
rural/ small town locations had higher average scale scores than their peers 
in central city schools. At grade 8, students from schools in urban fringe/ 
large town locations outperformed their peers from schools in rural/small 
town locations. 

► At grade 8, the percentages of students at or above the Basic and Proficient 
levels were higher among students from urban fringe/large town locations 
than among students from central city or rural/small town locations. At 
grade 4, the percentage of students at or above the Proficient level was 
higher among urban fringe/large town locations than among the other 
two groups. Both urban fringe/large town and rural/small town locations 
had higher percentages of students at the Basic level than did central 

city locations. 

Free/Reduced-Price School Lunch Program 

► At all three grades, students who were eligible for the federally funded 
Free/Reduced- Price School Lunch Program had lower average civics scale 
scores than students who were not eligible. 

Type of school 

► At all three grade levels, students attending nonpublic schools had higher 
average scale scores than their counterparts attending pubHc schools. The 
percentages at or above each of the achievement levels were also higher 
among students attending nonpublic schools than among students attending 
pubhc schools. 

Contexts for learning civics: Teacher characteristics 

► The majority of students at both grades 4 and 8 were taught civics by 
teachers at the Bachelor’s degree level (teachers in the twelfth grade did 
not receive a questionnaire). Forty-three percent of fourth graders and 
46 percent of eighth graders were taught by teachers who had Master’s or 
higher degrees. The fourth-grade students taught by Master’s-level teachers 
had higher scores than fourth-grade students taught by teachers with 
Bachelor’s degrees. 

► For both fourth- and eighth-grade student samples, the teacher’s 
undergraduate major in college was not related to student performance 
on the civics assessment. 
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► Fourth-grade students whose teachers had either advanced professional or 
regular teaching certification produced higher civics scores than students 
whose teachers had certification in the “temporary/provisional” category. 

► Students at both fourth and eighth grades generally had social studies 
teachers with three or more years’ experience. Only 11 percent of students 
at grade 4 and 10 percent at grade 8 had teachers with two years or less of 
experience. In the fourth grade only, students with teachers from the least- 
experienced category were outscored in the civics assessment by students 
whose social studies teachers had three or more years of experience. 

► Students at both fourth and eighth grades had teachers who rated 
themselves as well prepared in social studies instruction, classroom climate/ 
governance, classroom management, and the use of computers. However, 
teachers in both grades generally rated themselves as much less well 
prepared in using the voluntary national standards for civics/government 
or in using computer software for social studies instruction. 

► Over half of the teachers in both grades felt that they received “most” of the 
resources they need, while only one percent felt that they received “none” 
of the resources they need. 

► Across all three grades, 36 to 41 percent of students were in schools 
answering “yes” to whether computers were available to bring to classrooms 
when needed. In both grades 8 ind 12, students whose schools answered 
“yes” to this question outscored students whose schools answered “no.” 

► However, 67 percent of fourth graders and 63 percent of eighth graders 
were in classes where the teacher indicated “never or hardly ever” using 
computer software in class. In one fourth -grade category, students whose 
teachers indicated using computer software once or twice a month, 

had higher average scale scores than students in the “never or hardly 
ever” category. 

► Alrnpst. three-qu^ters (73 percent) of fourth graders were in classes where 
teachers indicated never or h^dly ever using the Internet for social studies. 
Internet usage appeared more common in the eighth grade, where only 

55 percent of students were in classes that never or hardly ever used it, while 
35 percent used it once or twice a month. In both the fourth and eighth 
grades, students using the Internet once or twice a month achieved higher 
average scale scores than students who never used the Internet. 
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Contexts for learning civics: 

Classroom and student variables 

► Over 70 percent of students at both grades 4 and 8 indicated that they had 
studied the U.S. Constitution and Congress in the current school year. 

► Generally, less than half the fourth- and eighth-grade students indicated that 
they had studied other countries’ governments or international organizations 
such as the United Nations (UN). 

► For both grades 4 and 8, the highest percentages of students were taught 
on a weekly basis with “traditional” instructional activities: using the social 
studies textbook; using quantitative data, charts or graphs; completing 
worksheets; hearing a teacher’s lecture; and using books, newspapers 

or magazines. 

► Instructional activities that were used on a weekly basis with low percentages 
of students included: use of computer software; writing a report of three or 
more pages; participating in debates or mock trials; and, writing letters on 
civic topics. 

^ Only 35 percent of students in the fourth grade and 43 percent in the 

eighth grade received group activities or projects on at least a weekly basis, 
however, in both grades, small-group activities were employed more 
commonly (53 percent in fourth grade and 52 percent in eighth grade) 
on a once-or-twice-a-month basis — and the assessment results in grade 8 
indicated that students in this frequency category outscored students who 
“never or hardly ever” participated in small-group activities. 

^ Teachers indicated how frequently they employed each of a wide range 
of classroom instructional activities with their social studies students. For 
all instructional activity categories at grade 4, the groups of students taught 
by teachers with three or more years of experience had higher civics scores 
than students taught by teachers with two years or less experience. This 
pattern of advantage for students of experienced teachers over students of 
less-experienced teachers by instructional activity did not appear, however, 
ini grade 8. 
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^ Among fourth graders, 58 percent of students were taught by teachers who 
had attended professional development activities in the past year. At grade 8, 
the comparable figure was 65 percent. At grade 4, students of teachers who 
attended workshops were taught less frequently using worksheets and more 
often using group activities and the “active” instructional techniques of 
debates, mock trials, and letter writing. Similarly, teachers at grade 8 who 
attended workshops were less likely to use textbooks and more likely to use 
extended reports, debates, and mock trials. 

^ At every grade there is a positive association between frequency of 

discussion of schoolwork at home and average civics scale scores. At least 
two-thirds of students across the three grades reported discussing 
schoolwork at home at least once a week. 

► More than half of grade 12 students indicated they did some volunteer work 
in their communities. Those who volunteered had higher civics scores than 
those who never volunteered. 

► Almost two-thirds of the twelfth-grade students indicated at least some 
hours worked each week at a job for pay. About a fifth (21 percent) of 
the students reported working 21 or more hours per week. Students who 
worked between 6 and 15 hours per week' had the highest scores on 

the assessment. 



CIVICS REPORT CARD • EXECUTIVE SUMAAARY 




The health of American constitutional democracy depends on the informed 
and active participation of the nation’s citizens in civic affairs. Yet, will the 
next generation of citizens have the skills and interest to fulfill its responsibUity.> 
Recent research into the civic attitudes of college students and adults reveals 
litde interest in pohtics or activism and poor knowledge of even such basic 
subjects as the United States Constitution. Although these findings do not 
augur well for the future, it is instructive that Americans still beheve that 
education for citizenship is one of the primary goals of education.' 

In rnost school districts around the country, students never take a course 
called civics; courses by that name were gradually phased out over much of the 
twentieth century. Today, civics concepts are primarily taught as part of social 
studies, history, and government classes. But, it is with the understanding that 
civics education — beginning early in a child’s educational career and 
continuing through high school — is critical to the survival of democratic 
government in this country that NAEP offers a unique assessment in civics at 
grades 4, 8, and 12. The results from the 1998 NAEP civics assessment 
presented in this Report Card can help answer the question: “Are the schools 
fulfilling their role as educators of good citizens.^” 

Overview of the 1998 National Assessment 
of Educational Progress (NAEP) 

This report is wntteri for a variety of audiences — policymakers, parents, 
teachers, and concerned citizens. The results reported here can provide 
important information for them to consider in discussing and making decisions 
about how best to prepare the nation’s students to become concerned and 
active citizens. 



^ The results of recent research are summarized by Branson, M.S. (1998) in The role of civic education. 
[On-line], Available: http://www.civiced.org/articles_role,html. 

Branson, M.S. ( 1994). What does research on political attitudes and behavior tell us about the need for 
improving education for democracy? Paper presented at the International Conference on Education for 
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The NAEP is the only nationally representative and continuing assessment 
of what students in the United States know and can do in various subjects. 
NAEP is authorized by Congress and directed by the National Center for 
Education Statistics (NCES) of the U.S. Department of Education. The 
National Assessment Governing Board (NAGB), an independent body, provides 
policy direction for NAEP. 

Since it was authorized by Congress in 1969, NAEP’s mission has been 
to collect, analyze, and present reliable and valuable information about what 
students know and can do. Both public and private school students in grades 4, 
8, and 12 are sampled and assessed on a regular basis in core academic 
subject areas. 

All NAEP assessments are based on a content framework developed through 
a national consensus process that involves teachers, curriculum experts, parents, 
and members of the general pubUc. The NAEP Civics Framework^ was newly 
written for the 1998 assessment, as were all of the assessment questions. 
Therefore, it is not possible to compare results of student achievement in civics 
from the 1998 assessment with those of earlier assessments. However, a separate 
trend study was conducted in 1998 using questions from the previous NAEP 
civics assessment conducted in 1988. Results from the trend study wiU appear in 
a separate report. 

The 1998 civics assessment was administered to national samples of fourth, 
eighth, and twelfth graders. At fourth grade 5,948 students were assessed; at 
eighth grade, 8,212; and at twelfth grade, 7,763. 

This report describes the results of the NAEP 1998 civics assessment. The 
assessment consisted of a combination of multiple-choice and constructed- 
response (open-ended) questions. Each student participating in the assessment 
received two 25-minute sections of questions. There were 15 questions in each 
section at grade 4, 19 questions in each section at grade 8, (with the exception 
of one section with 18 questions); and 19 questions in each section at grade 12. 

Framework for the 1 998 Assessment 

The NAEP Civics Framework established the objectives for the 1998 civics 
assessment and provided the specifications that guided development of the 
assessment instrument. The framework was developed for the National 
Assessment Governing Board (NAGB) through a national consensus process 



^ National Assessment Governing Board. (1996). Civics framework for the 1998 National Assessment of 
Ediscistional Projjiress. Washington, DC: Author. 
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conducted by the Council of Chief State School Officers in conjunction with 
the Center for Civic Education and the American Institutes for Research. 

It benefited from the input of many and diverse perspectives. The steering 
committee that oversaw the fi:amework development was comprised of 
representatives firom education and policy organizations, business, and 
government. The planning committee that wrote the framework Was made 
up of teachers and teacher educators, scholars, curriculum experts, and 
assessment specialists. In addition, input was solicited and received from 
educators, scholars, students, and the general public. 

Development of the framework rested on two important ideas. First, 
the preservation of American constitutional democracy depends upon a well- 
educated citizenry participating actively in public afifeirs. To be well educated 
means not only to have knowledge about government and society, but also to 
possess the skills and civic dispositions necessary for effective participation. 
Second, although families, religious institutions, and other influences have an 
important role in shaping individuals’ civic character, the schools “bear a special 
and historic responsibility for the development of civic competence and civic 
responsibility. Schools can and should provide effective civic education through 
both formal and informal means from the earliest grades through high 
school.”^ Therefore, as embodied by the framework, the goal of the NAEP 
civics assessment is to measure how well American youth are being prepared 
to meet their citizenship responsibilities. 

The design of the 1998 assessment reflects this goal. The framework 
specifies three interrelated components which, taken together, reflect broad 
civic competency: 

^ Knowledge 

► Intellectual and participatory skills 
^ Civic dispositions 

Each assessment exercise has a knowledge and intellectual skills component. 
A portion of the exercises also measures participatory skills and/or civic 
dispositions. 



^ Nation^ Assessment Governing Board. (1996). Civics framework for the 1998 National Assessment of 
EduccwtiGttal FrG^ffTcss. i;^. 7). Washington, DC: Author. 
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The framework’s civic knowledge component draws heavily on the 1994 
National Standards for Civics and Government developed by the Center for 
Civic Education and covers the broad range of content that forms the basis of 
civic understanding."^ Civic knowledge is divided into five categories that are 
expressed as questions. That the framework poses the knowledge component as 
questions reflects the position that civic knowledge encompasses not just factual 
knowledge but a broader and deeper understanding of the meaning of 
citizenship. The categories are: 

I. What are civic life, politics, and government? 

II. What are the foundations of the American political system? 

III. How does the government established by the Constitution embody the 
purposes, values, and principles of American democracy? 

rV. What is the relationship of the United States to other nations and to world 
affairs? 

V. What are the roles of citizens in American democracy? 

The intellectual and participatory skills component covers those skills of 
mind and action that allow individuals to apply civic knowledge to good effect. 
The framework distinguishes three types of intellectual skills, although clearly 
there is some degree of overlap among them: 

► Identifying and describing 
^ Explaining and analyzing 

► Evaluating, taking, and defending positions 

The framework seeks to measure students’ participatory skills because, 
ultimately, civic knowledge and intellectual skills are most beneficial when 
applied to civic participation. Since NAEP cannot directly assess civic participation, 
the framework specifies that test questions be designed to measure whether 
students can identify participatory skills, recognize their purpose, explain how to 
use them, or specify how. best to achieve desired results by using particular skills. 



^ Center for CMc Education. (1994). Natiofinl standards civics and aovernment. Calabasas., CA: Author. 
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The third framework component, civic dispositions, “refers to the traits of 
private and public character essential to the preservation and improvement of 
American constitutional democracy.” The framework identifies five dispositions: 

► Becoming an independent member of society 

^ Assuming the personal, political, and economic responsibilities of a citizen 
^ Respecting individual worth and human dignity 

^ Participating in civic affairs in an informed, thoughtful, and effective manner 
^ Promoting the healthy functioning of American constitutional democracy 

As with participatory skills, it is not within the scope of NAEP to assess 
students’ civic dispositions direcdy. Therefore, certain assessment exercises are 
designed to measure students’ knowledge and understanding of the importance 
of civic dispositions. 

The new framework for the 1998 civics assessment differs from the Civics 
Objectives for the 1988 assessment in two principal ways.^ First, whereas the 
1988 Objectives focused on student understanding of United States government 
and politics, the 1998 current framework defines civic knowledge more broadly. 
Reflecting recerit scholarship, the framework incorporates the idea that the 
healthy functioning of democratic society requires participation not only in 
those organizations and activities directly tied to governmental politics but also 
in the wide range of voluntary, nongovernmental organizations and activities — 
unions, religious organizations, clubs, charities, service organizations, and so 
on — that constitute what is known as “civil society.”^ The second major 
difference between the 1998 assessment and its predecessor is the inclusion of 
open-ended (i.e., constructed-response) exercises. In 1988 the assessment at 
grade 4 consisted entirely of multiple-choice questions, and the assessments at 
grades 8 and 12 included only one open-ended question. In 1998, as shown in 
Table i.l, significant portions of the assessment time at all three grades were 
allotted to open-ended questions. 



^ National Assessment of Educational Progress. (1987^. Civics: United States government and policy objectives: 
1988 assessment, Princeton, NJ: Educational Testing Service. 

^ Bahmueller, C.F., Quigley, C.N. (Eds.). (1991). CIVITAS: A framework for civic education. Calabasas, CA: 
Center for Civic Education. 
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Percentage of civics assessment time by item format, 
grades 4, 8, and 12: 1998 








Grade 4 


Grade 8 


Grade 12 


Multiple choice 


53 


61 


61 


Short constructed response 


29 


27 


30 


Extended constructed response 


18 


12 


9 



SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 



The Civics Assessment Instruments 

To ensure that the civics assessment instruments conformed as closely as 
possible to the framework specifications, the development of the 1998 
assessment was guided by a special committee (see Appendix C) of civics 
teachers and teacher educators, some of whom had served on the framework 
planning committee. All components of the assessment were also evaluated for 
curricular relevance, developmental appropriateness, and fairness concerns. Five 
hundred fifry-five questions were field-tested in 1997 and their performance 
analyzed. Test developers then used the best available questions to meet the 
criteria established for the 1998 assessment. 

Tables i.2 and i.3 show the distribution of assessment time across areas 
of civic knowledge and intellectual skills, respectively. Every effort was made 
to create a question pool that met the framework specifications for the 
distribution of exercises by content and intellectual skills. However, the 
additional need to balance the pool by exercise format (multiple-choice and 
open-ended), include questions based on a variety of textual and graphical 
stimuli, measure participatory skills and civic dispositions, and use questions that 
had performed best during the field test meant that the distribution of exercises 
in the 1998 assessment instruments differs somewhat from the distribution 
specified in the framework. 
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Table i.2 








Percentage distribution of civics assessment time by content area, 
grades 4, 8, and 12: 1998 
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Grade 4 Grade 8 Grade 1 2 



I. What ore civic life/ politics, 

and government? 21 8 5 



II. what are the foundations 
of the American political system? 


22 


28 


23 


III. How does the government established 
by the Constitution embody the purposes, 
values, and prindples of American democracy? 


15 


26 


29 


IV. What is the relationship of the United States 
to other notions and to world affairs? 


10 


14 


20 


V. What are the roles of 

citizens in American democracy? 


32 


24 


23 



SOURCE: Notionol Center for Educotion Statistics, 
Notionol Assessment of Educotionol Progress (NAEP), 
1 998 Civics Assessment. 




Percentage distribution of civics assessment time by intellectual skills, 
grades 4, 8, and 12: 1998 





Grade 4 


Grade 8 


Grade 12 


Identifying and describing 


33 


29 


18 


Explaining and analyzing 


37 


38 


44 


Evaluating, taking, and defending a position 


30 


33 


38 



SOURCE: Notionol Center for Educotion Statistics, 
Notionol Assessment of Educotionol Progress (NAEP), 
1 998 Civics Assessment. 
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For the 1998 assessment, the exercise pools at each grade were divided into 
25-minute blocks; six at fourth grade, and eight blocks each at eighth grade 
and twelfth grade. Each student participating in the assessment received a test 
booklet that contained two blocks of questions. Although the framework 
distribution requirements refer to the entire exercise pool at each grade, the 
blocks were assembled to be roughly similar in terms of difficulty and exercise 
format. Every block also contained questions from each of the content and 
intellectual skills categories, but not in equal proportions. 

As part of the assessment, students answered general background questions 
that asked them to identify their gender, race/ethnicity, parents’ highest level 
of education, and other fiictors such as how often they studied civics or social 
studies. These background questions were given in separately timed sections. 
Results from these questions and other background questions given to the 
teachers and school administrators of participating students are discussed in 
Chapters 4 and 5. 

Description of School and Student Samples 

The NAEP 1998 civics assessment was conducted nationally at grades 4, 

8, and 12. A representative sample of public and nonpublic school students, 
selected through stratified random sampling procedures, was assessed. Thus, 
the results presented in this report are based on representative samples of 
students. Each selected school that participated in the assesisment, and each 
student assessed, represents a portion of the population of interest. As a result, 
after adjustment for student and school nonresponses, the findings presented are 
representative of all fourth, eighth, and twelfth graders in the nation and in the 
four regions analyzed (Northeast, Southeast, Central, West). 

More details on the nature of the accommodations offered to students 
with disabilities and students with limited English proficiency, and the number 
of students who received such accommodations, are given in Appendix A. 
Appendix A also contains information on sample sizes and participation rates 
for the assessment. 

In an effort to expand inclusion in NAEP, the 1998 civics assessment, for 
the first time, offered testing accommodations to students with disabilities and 
to students with limited proficiency in English. Some of the accommodations 
provided were small groups, extended time, and one-on-one administration. A 
total of 3 percent of fourth-grade students, 3 percent of eighth-grade students, 
and 1 percent of twelfth-grade students were assessed with accommodations. 







8 



25 



CIVICS REPORT CARD • INTRODUCTION 



Scoring the 1998 Civics Assessment 

All student responses to open-ended exercises were scored by specially trained 
professional raters. Scoring guides were developed for each open-ended exercise 
in the assessment. Responses to most open-ended questions were scored on 
either a 3- or 4-point scale, depending on their complexity. Question authors 
drafted initial scoring guides to reflect the purpose of the exercise. Generally a 
scoring guide contained a rubric establishing the criteria for receiving different 
score levels plus a list of credited responses outlining which answers would and 
would not receive credit. Test developers revised and refined the scoring guides 
at several stages before the field test, after the field test, and again before the 
1998 assessment. The guides were also reviewed by the civics development 
committee prior to both the field test and the 1998 assessment, and by NAGB 
and NCES staff and outside reviewers during the two clearance reviews. During 
the scoring process, 25 percent of all student responses were read by a second 
rater to ensure a high degree of consistency between raters. 

A more detailed description of the NAEP scoring procedure is given in 
Appendix A. 

Reporting the Civics Assessment Results 

Because of the nature of the NAEP assessment, which selects a representative 
sample of students in order to survey the nation, and the breadth of the field 
of civics, each student participating cannot be expected to respond to all of 
the exercises in the assessment. That would impose an unreasonable burden 
on students and their schools. Thus, each student was administered a portion 
of the assessment, and data were combined across students to report on the 
achievement of fourth, eighth, and twelfth graders and on the achievement 
of subgroups of students (e.g., subgroups defined by gender or level of 
parental education). 

Student responses to the civics questions were analyzed to determine the 
percentage of students responding correcdy to each multiple-choice question 
and the percentage of students responding at each of the score levels for 
constructed-response questions. Item response theory (IRT) methods were 
used to produce an overall scale for each of the grades — 4, 8, and 12. Results 
presented in this report are based on this overall scale. 
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For each grade, the range of the scale was 0 to 300, and the national average 
was set at 150. Although the scale-score ranges are identical across grades, the 
scale was derived independently for each grade. Therefore, average scale scores 
cannot be compared across grades. For example, equal scale scores on the grade 
4 and grade 8 scales do not imply equal levels of civics achievement. However, 
this scale does make it possible to compare civics scale scores for the nation for 
subgroups of students at a particular grade. (Additional details of the scaling 
procedures can be found in Appendix A.) 

The average scale score provides information on what students know and 
can do. In addition to the NAEP civics scale, results are also reported using the 
civics achievement levels as authorized by the NAEP legislation and adopted 
by NAGB. The achievement levels are performance standards based on the 
collective judgments of experts about what students should know and be able to 
do. The levels were developed by a broadly representative panel that included 
teachers, education specialists, ^d members of the general public. For each 
grade tested, NAGB has adopted three achievement levels: Basic, Proficient, 
and Advanced. For reporting purposes, the achievement level cut scores for 
each grade are placed on the NAEP civics scale, resulting in four ranges: below 
Basic, Basic, Proficient, and Advanced. 

The Setting of Achievement Levels 

TheT988 NAEP legislation that created the National Assessment Governing 
Board directed the Board to identify “appropriate achievement goals ... for 
each subject area” that NAEP measures.^ The 1994 NAEP reauthorization 
reaffirmed many of the Board’s statutory responsibilities, including “developing 
appropriate student performance standards for each age and grade in each 
subject area to be tested under the National Assessment.”® To follow this 
directive and achieve the mandate of the 1988 statute “to improve the form 
and use of NAEP results,” the Board undertook the development of student 
performance standards (called “achievement levels”). Since 1990, the Board 
has adopted achievement levels for mathematics, reading, U.S. history, 
world geography, and science, and, most recently, for the 1998 civics and 
writing assessments. 



^ Public Law 100-297. (1988). National Assessment of Educational Progress Improvement Act 
(20 use 1221). Washington, DC. 

® Public Law 103-382. (1994). Improving America’s Schools Act (20 USC 9010). Washington, DC. 
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The Board defined three levels for each grade: Basic, Proficient, and 
Advanced. The Basic level denotes partial mastery of the knowledge and skills 
that are fundamental for proficient work at a given grade. The Proficient level 
represents solid academic performance. Students reaching this level demonstrate 
competency over challenging subject matter. The Advanced level signifies 
superior performance at a given grade. For each grade, the levels are cumulative; 
that is, abilities achieved at the Proficient level presume mastery of abilities 
associated with the Basic level, and attainment of the Advanced level presumes 
mastery of both the Basic and Proficient levels. Figure i.l presents the policy 
definitions of the achievement levels that apply across grades and subject areas. 
(Specific descriptions of civics achievement for the levels at each grade are 
presented in Chapter 1.) Adopting three levels of achievement for each grade 
signals the importance of looking at more than one standard of performance. 
The Board believes, however, that all students should reach the Proficient level; 
the Basic level is not the desired goal, but rather represents partial mastery that 
is a step toward Proficient. 



Figure i.l 




Achievement level policy definitions 
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Basic 



Proficient 



Advanced 



This level denotes partial mastery of prerequisite knowledge and skills 
that are fundamental for proficient work at each grade. 

This level represents solid academic performance for each grade assessed. 
Students reaching this level have demonstrated competency over 
challenging subject matter, including subject-matter knowledge, 
application of such knowledge to real-world situations, and analytical 
skills appropriate to the subject matter. 

This level signifies superior performance. 
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The achievement levek in this report were adopted by the Board based on 
a standard-setting process designed and conducted under a contract with ACT, 
Inc. To develop these levels, ACT convened a cross-section of educators and 
interested citizens from across the nation and asked them to judge what 
students should know and be able to do relative to the view of civics reflected 
in the NAEP assessment framework for civics. This achievement-level-setting 
process was reviewed by an array of individuals, including policymakers, 
representatives of political organizations, teachers, parents, and other members 
of the general public. Prior to adopting these levels of student achievement, 
NAGB engaged a large number of persons to comment on the recommended 
levels and to review the results. 

The results of the achievement-level-setting process, after NAGB approval, 
are a set of achievement level descriptions and a set of achievement level cut- 
points on the 300-point NAEP scale for civics, as well as a set of exemplars 
of student performance at each achievement level. The cut points are the 
scores that define the boundaries between below Basic, Basic, Proficient, 
and Advanced performance at grades 4, 8, and 12. Further details of the 
achievement-level-setting process will be available in the forthcoming NAEP 
1998 Technical Report. 

The Developmental Status of Achievement Levels 

The 1994 NAEP reauthorization law requires that the achievement levels be 
used on a developmental basis until the Commissioner of Education Statistics 
determines that the achievement levels are “reasonable, valid, and informative 
to the public.”^ Until that determination is made, the law requires the 
Commissioner and the Board to make clear the developmental status of the 
achievement levels in all NAEP reports. 

In 1993, the first of several congressionally mandated evaluations of the 
achievement-level-setting process concluded that the procedures used to set the 
achievement levels were flawed, and that the percentage of students at or above 
any particular achievement level cut point may be underestimated.*® Others 
have critiqued these evaluations, asserting that the weight of the empirical 
evidence does not support such conclusions.** 

^ The Improving America’s Schools Act of 1994 (20 USC 9010) requires that the Commissioner base his or 
her determination on a congressionally mandated evaluation by one or more nationally recognized 
evaluation organizations, such as the National Academy of Education (NAE) or the National Academy of 
Sciences (NAS). 

Umted Stat« General Accounting Office. (1993). Education achievement standards: NAG B’s approach yields 
misleading interpretations. U.S. General Accounting Office Report to Congressional Requestors. 
Washington, DC: Author. 

National Academy of Education. (1993). Setting performance standards for achievement: A report of the 
National Academy of Education Panel on the evaluations of the NAEP Tnal State Assessment: An evaluation 
of the 1992 achievement levels. Stanford, CA: Author 

Cizek, G. (1993). Reactions to National Academy of Education report. Washington, DC: National 
Assessment Governing Board. 

Kane, M. (1993). Comments on the NAEP evaluation of the NAGB achievement levels. Washington, DC: 
National Assessment Governing Board. 
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In response to the evaluations and critiques, NAGB conducted an additional 
study of the 1992 achievement levels in reading before deciding to use those 
levels for reporting 1994 NAEP results.*^ When reviewing the findings of this 
study, the National Academy of Education (NAE) Panel expressed concern 
about what it saw as a “confirmatory bias” in the study and about the inability 
of this study to “address the panel’s perception that the levels had been set 
too high.”*^ 

In 1997, the NAE Panel summarized its concerns with interpreting NAEP 
results based on the achievement levels as follows: 

First, the potential instability of the levels may interfere with the 
accurate portrayal of trends. Second, the perception that few American 
students are attaining the higher standards we have set for them may 
deflect attention to the wrong aspects of education reform. The public has 
indicated its interest in benchmarking against international standards, 
yet it is noteworthy that when American students performed very well on 
a 1 991 international reading assessment, these results were discounted 
because they were contradicted by poor performance against the possibly 
flawed NAEP reading achievement levels in the following year.'^* 

The NAE Panel report recommended “that the current achievement levels 
be abandoned by the end of the century and replaced by new standards 
The National Center for Education Statistics and the National Assessment 
Governing Board have sought and continue to seek new and better ways to set 
performance standards on NAEP For example, NCES and NAGB jointly 
sponsored a national conference on standard setting in large-scale assessments, 
which explored many issues related to standard setting.*^ Although new 
directions were presented and discussed, a proven alternative to the current 
process has not yet been identified. The Acting Commissioner of Education 
Statistics and NAGB continue to call on the research community to assist in 
finding ways to improve standard setting for reporting NAEP results. 



American College Testing. (1995). NAEP reading revisited: An evaluation of the 1992 achievement level 
descriptions. Washington, DC; National Assessment Governing Board. 

Nation^ Academy of Education. (1996). Reading achievement levels. In Quality and utility: The 1994 
Trial State Assessment in reading. The fourth report of the National Academy of Education Panel on the 
evaluation of the NAEP Tnal State Assessment. Stanford, CA: Author. 

National Academy of Education. (1997). Assessment in transition: Monitoring the nation^s educational 
progress {p. 99). Mountain View, CA: Author. 

National Assessment Governing Board and National Center for Education Statistics. (1995). Proceedings of 
the Joint Conference on Standard Setting for Large-Scale Assessments of the National Assessment Governing 
Board (NAGB) and the National Center for Education Statistics (NCES). Washington, DC: Government 
Printing Office. 
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The most recent congressionally mandated evaluation conducted by the 
National Academy of Sciences (NAS) relied on prior studies of achievement 
levels, rather than carrying out new evaluations, on the grounds that the process 
has not changed substantially since the initial problems were identified. Instead, 
the NAS Panel studied the development of the 1996 science achievement levels. 
The NAS Panel basically concurred with earlier congressionally mandated 
studies. The Panel concluded that “NAEP’s current achievement-level-setting 
procedures remain fundamentally flawed. The judgment tasks are difficult and 
confusing; raters’ judgments of different item types are internally inconsistent; 
appropriate validity evidence for the cut scores is lacking; and the process has 
produced unreasonable results.”’* 

The NAS Panel accepted the continuing use of achievement levels in 
reporting NAEP results only on a developmental basis, until such time as better 
procedures can be developed. Specifically, the NAS Panel concluded that 
“... tracking changes in the percentages of students performing at or above 
those cut scores (or in fact, any selected cut scores) can be of use in describing 
changes in student performance over time.”'^ 

The National Assessment Governing Board urges all who are concerned about 
student performance levels to recognize that the use of these achievement levels is 
a developing process and is subject to various interpretations. The Board and the 
Acting Commissioner of Education Statistics believe that the achievement levels 
are useful for reporting on trends in the educational achievement of students in 
the United States. In fact, achievement level results have been used in reports by 
the President of the United States, the Secretary of Education, state governors, 
legislators, and members of Congress. The National Education Goals Panel and 
government leaders in the nation and in more than 40 states use these results in 
their annual reports. 

Based on the congressionally mandated evaluations so far, the Acting 
Commissioner agrees with the recommendation of the National Academy 
of Sciences to exercise caution in the use of the current achievement levels. 
Therefore, the Acting Commissioner concludes that these achievement levels 
should continue to be considered developmental and should continue to be 
interpreted and used with caution. 



’ ^ (Eds.). (1999). Grading the nation’s report card: Evaluating 

NAEP and tranformtn^ die assessment of educational progress. Committee on the Evaluation of National 
Assessmehte of Educational Progress, Board on Testing and Assessment, Commission on Behavioral and 
Social Sciences and Education, National Research CouncU. (p. 182), Washington, DC: National Academy 
Press, ^ 

Ibid., page 176, (For an opposing viewpoint see Hambleton, R,IC, Brennan, R,L., Brown, W, Dodd, B., 
Forsyth, R.A,, Mehrens, WA,, Nellhaus, J., Reckase, M., Rindone, D., van der Linder, WJ., & Zwick, R. 
(1999). A response to ^Setting reasonable and useful performance standards^ in the National Academy of 
Sciences’ Grading the nation’s report card. Amherst, Massachusetts: University of Massachusetts Laboratory 
of Psychometric and Evaluative Research.) 
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interpreting NAEP Results 

The average scores and percentages presented in this report are estimates 
because they are based on samples rather than testing an entire population. 

As such, the results are subject to a measure of uncertainty, reflected in the 
standard errors of the estimates. Also reflected in the standard errors are errors 
of measurement that are associated with any measurement instrument. The 
standard errors for the estimated scale scores and percentages provided 
throughout this report are provided in Appendix B. 

The differences between scale scores or percentages discussed in the 
following chapters take into account the standard errors associated with the 
estimates. The comparisons are based on statistical tests that consider both the 
magnitude of the difference between the group average scores or percentages 
and the standard errors of these statistics. Throughout this report, differences 
are defined as significant when they are significant from a statistical perspective. 
This means that observed differences are unlikely to be due to chance factors 
associated with sampling variabihty. The term “significant” is not intended to 
imply a judgment about the absolute magnitude or educational relevance of 
the differences. It is intended to identify statistically dependable population 
differences in order to help focus subsequent dialogue among policymakers, 
educators, and the public. All differences reported are significant at the .05 
level, with appropriate adjustments for multiple comparisons. 

Cautions in Interpretation 

The reader is cautioned against using the NAEP results in this report to make 
simple causal inferences related to subgroup performance, to the relative 
effectiveness of public and nonpublic schools, or to other educational variables 
discussed in this report. A relationship that exists between performance and 
another variable does not reveal the underlying cause of that relationship, 
which may be influenced by a number of other variables. Differences in civics 
performance may reflect a range of socioeconomic and educational factors 
not discussed in this report or addressed by the NAEP assessment program. 
Similarly, differences between public and nonpublic schools may be better 
understood by considering such factors as composition of the student body 
and parental involvement. Finally, differences in civics performance may reflect 
not only the effectiveness of education programs, but also the challenges posed 
by economic constraints and student demographic characteristics. 
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Average Scale Score and 
Achievement Level Results 
for the Nation 




Overview 

This chapter presents overall results of the 1998 NAEP civics assessment for all 
fourth, eighth, and twelfth graders across the nation. Chapters 2 and 3 provide 
results for subgroups of students. As discussed in the Introduction to this 
report, student performance on the civics assessment is described in two ways: 

1) in terms of average scores on the NAEP civics scale, and 2) in terms of the 
percentages of students attaining the three levels of achievement in civics. The 
NAEP civics scale ranges from 0 to 300. The three civics achievement levels are 
Basic^ Proficienty and Advanced. 

This chapter also presents sample questions and student responses from 
the assessment. These questions and responses were selected to exemplify 
performance within each achievement level range. Three questions are presented 
for each grade assessed — one at each achievement level. 

Average Scale Score Results for the Nation 

The NAEP civics assessment provides a measure of students’ knowledge and 
understanding of civics. The average performance of students in each grade 
assessed is expressed in terms of the average score on the NAEP civics scale. 

As noted above, the civics scale ranges from 0 to 300, and the national average 
at each grade is 150. The NAEP civics scale can be used to compare the 
performance of subgroups of students within a grade (for example, male 
eighth graders compared to female eighth graders). Within-grade scale score 
comparisons of subgroups of students are presented in Chapter 2 of this report. 

• ji i ! i 

The scale does not allow for comparisons of performance across grades (for 
example, fourth graders compared to eighth graders) because each grade was 
scaled separately. Additional information about the scaling procedures for the 
1998 NAEP cmcs assessment can be found in Appendix A of this report and 
in the forthcoming NAEP 1998 Technical Report. 
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Table 1.1 presents the civics scale scores for students at the 10*, 25*, 50*, 
75*, and 90* percentiles at each grade. These data provide some indication of 
the range of student performance, from lower performance (10* percentile) to 
higher performance (90th percentile). 




Civics scale scores by percentiles: 1998 



THE NATION'S 
REPORT 
CARO 







SOURCE: NoHond Center br Education Statistics, National Assessment of Educational 
Progress (NAEP), 1 998 Civics Assessment. 



Achievement Level Results for the Nation 

In addition to describing student performance in terms of average scale scores, 
the assessment results can be examined according to students’ attainment of 
the three civics achievement levels as authorized by the NAEP legislation* and 
as adopted by the National Assessment Governing Board (NAGB). Viewing 
students’ performance from this perspective provides some insight into the 
adequacy of students’ knowledge and skills and the extent to which they 
achieved expected levels of performance. 

In 1999, NAGB reviewed and adopted the recommended civics achievement 
levels of Banc, Projicient, and Advanced that, for reporting purposes, are placed 
on the NAEP civics scale resulting in the four ranges of performances cited 
earlier (i.e., below Basic, Basic, Proficient, and Advanced). Figures 1.1 through 
1.3 present in bold t^e the specific descriptions of civics achievement levels at 
each* grade. In the description of each achievement level, the portion in regular 
type presents a summary of the complete description of that level. The numbers 
in parentheks are the scale scores marking the boundary for each level on the 
0-300 scale. 



* The Improving America’s Schools Act of 1994 (20 USC 9010) requires that the National Assessment 
Governing Board develop “appropriate student performance levels” for reporting NAEP results. 
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Civics achievement levels, grade 4: 1998 



Bask Fourth-grade students perfornring at the basic level should have an understonding of what government is and 
(136) what it does, end they should be oble to identify some things that government is not allowed to do. These 
students should have some understanding of the foundotions of the Amerkon political system. In the context 
of their school ond community, they should understood rules and lows, rights ond responsibilities, and woys to 
portidpote in governing. These students should know tbot the world is divided into mony countries. 

Fourtli-grade students performing at the basic level should have some understanding of what government is and what it does, and they 
should be able to identify some things that government is not allowed to do. They should be able ta explain purposes of rules in the 
school and the community, and to describe what happens when people break laws. These students should understand how national 
holidoys and symbols such as the flag, the Statue of Uberty, and the Fourth of July reflect shared American values, and they should be 
oble to identify different types of diversity In American society. They should be able to describe ways to settle disagreements or conflicts 
peocefully. They should be able to name the president and their state governor and to identify rights and responsibilities of a citizen. 

They should know some ways that students can participate in governing their school and community, and they should be able ta describe 
quolities of a good leoder. Finally, these students should know that the world Is divided into many countries. 

ProHdeat Fourth-grade students performing at the proficient level should hove a good understonding of what the 
(1 American government does and of why it is not oUowed to oct in certain woys. These students should hove an 
age-oppropriate understanding of the foundations of the American politicol system. They should understond 
purposes of lows, ways shared beliefs unify Americans, what it means to be a dtizeiv and rights and 
responsibilities of citizens, and the idea of public participation in governing. These students should be able 
to describe woys in which countries interact with one another. 

Fourth-grade students performing at the proficient level should have a good understanding of what the American government does 
ond of why it is not ollowed to act in certain ways. They should be oble to explain why we have laws. These students should be able 
to recognize diversity in American society and that Americans are united by shared beliefs and principles. They should know that the 
Constitution and the Declaration of Independence are founding documents of American democracy. They should be able to explain how 
people moke decisions obout the ways they live together in a democracy and how groups in schools and communities can manage 
conflict peacefully. They should know what it means to be a citizen of their state and the nation, and they should be able to distinguish 
between rights ond responsibilities of citizens. They should understand why it is important for people ta participate in governing their 
school ond community. Finally, these students should be able to describe ways in which countries Interact with one another. 

Advaatad Fourth-grade students performing at the advanced level should understand and be able to explain some 
(215) purposes of government. When given oge-appropriote examples, they should recognize differences between 
power and authority and between limited ond unhmited government. They should be oUe to exploin the 
importonce of shored values in Americon democracy, to identify woys citizens con participate in governing, and 
to understand thot with rights come responsibilities. They should be oble to explain how nations benefit when 
they resolve conflicts peacefully. 

Fourth-grade students performing at the advanced level should understand and be able to explain same purposes of government. They 
should recognize differences between power and authority when given examples and should understand differences between limited and 
unlimited gavernment. These students should be able to explain why it is important that citizens share a commitment to the values of 
Americon demoaocy, and they should be aware of the benefits and challenges of both unity and diversity in American society. They 
should be able to distinguish between services provided by local and state levels of government. These students should be able to 
desaibe how government can make it possible for people to accomplish goals they could not achieve alone. They should be able to 
Identify ways In which citizens can keep track of their government's actions, and they should understand the connection between rights 
and responsibilities of a citizen. Finally, they should be able to explain how nations benefit when they resolve conflicts peacefully. 



SOURCE: National Assessment (Governing Board. Policy adopted May 15, 1999 at quarterly meeting. Washington, DC. 
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Figure 1.2 



Civics achievement levels, grade 8: 1998 
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Bask Eighth-grade students performing at the basic level should have some understanding of competing ideos ohout purposes 

(134) of government, and they should he able to describe advontages of limited government. They should be able to define 

government, constitution, the rule of low, and politics. Tliey should be able to identify the fundomental principles of 
American democracy ond the documents from which they originate, and they should understand the importance of a 
shored commitment to the core values of American democracy. They should recognize the components of the politicol 
process and understand personal, political, ond economic rights ond responsibilities. They should be able to describe the 
purposes of some international orgonizotions. 

Eighth-grade students performing at the basic level should hove some understanding of competing ideas about purposes of government, and they 
should be able to describe advantages of limited government. They should be able to define what is meant by government, constitution, the rule 
of low, and politics. These students should be able to identify fundamental principles and values of American democracy, such os federalism, the 
separation of powers, checks and balances, government by the consent of the governed, and individual rights. They should understand that the 
Declaration of Independence and the United States Constitution including the Bill of Rights and other Amendments ore sources of these ideas. These 
students should be able to explain why it is important that citizens shore the values and principles expressed in the nation's core documents, and 
they should understand functions of elections, political parties, and interest groups in o democratic society. They should know that American 
citizenship is attained by birth or through naturalization. They should be able to identify personal, political, and economic rights of Americans and 
should understand the responsibilities that these rights imply. Finally, these students should be able to desaibe purposes of international 
organizations to which the United States belongs. 

Pralidettt Eighth-grade students performing at the proficient level should understand and be able to explain purposes that 
(178) government should serve. These students should have a good under stonding of differences between government ond 
civil sodety and of the importance of the rule of law. They should recognize discreponcies between American ideals and 
reality and be oble to describe continuing efforts to address them. They should understand the seporotion ond shoring 
of powers among branches of government and between federal and stote governments, ond they should be able to 
exploin how dtizens influence government. They should be oble to describe events within the United Stotes ond other 
countries thot have internationol consequences. 

Eightfhgrode students performing at the proficient level should hove o good understanding of purposes that government should serve, and they 
should be able to explain why government should serve those purposes. These students should understand differences between government and 
civil society, and they should be able to explain the importance of the rule of low. They should be able to point out ways in which ideals expressed 
in the nation's core documents differ from reality and to identify ways in which these differences continue to be addressed. They should be able to 
explain how and why legislative, executive, and judicial powers ore separate, shared, and limited in the American constitutional government, and 
they should understand how and why powers ore divided and shared between the notional and state governments. They should be able to discuss 
ways that citizens con use the political process to influence government. These students should be able to provide simple interpretations of non- 
text-based information, like mops, charts, tobies, graphs, and cartoons. Finally, these students should be able to describe events in the United States 
that hove influenced other nations, os well os events in other nations that hove offected American policy. 

Adwaasail Eighfh-grode sfudmfs performing of the advanced level should have a developed understanding of how dvtl sodety 

(213) helps to mointain limited government ond why the rule of low is important. These students should hove o deor 

understonding of issues in which democrotic volues ore in conflict and of post efforts to address the disaepandes 
between Americon ideals ond reolity. They should understand how dtizens can monitor and influence government 
ond how responsible citizens support demoaacy. They should recognize the hnpoct of American demoaocy on other 
countries, os well os other countries' hnpoct on American politics ond sodety. 

Eightihgrade students performing at the advanced level should hove o developed understonding of why civil society ploys o key role in maintaining 
0 limited government and of the importance of the rule of low in civil society and government. These students should be able to toke positions on 
issues in which fundomentol values ore in conflict — liberty and equality, individual rights and the common good, and majority rule and minority 
rights, for example — and they should be able to defend their positions. They should be oble to evaluate results of post efforts to address 
discrepancies between American ideals and notional reality and to explain how citizens con monitor and influence locol, state, and notional 
government. These students should understand how lows con achieve purposes of American constitutional government, such os promoting the 
common good and protecting rights of individuals. They should understand how civic dispositions such os civility, tolerance, and respect for low 
promote the healthy functioning of American constitutional democracy. Finally, these students should understand the impact of American democracy 
on other countries, os well os the impact of other countries on American politics and society. 

SOURCE: NotlOPot rrowAminn Rrvirrl Pnliry nrlrvntArl Mny 1 .S 1999 nt qiinrterlv meetioa. Washinaton. CXI. 
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Civics achievement levels, grade 12: 1998 

Basic Twelfth-grode students performing at the bosk level should have an understanding of what is meont by civil society, constitu- 

(139) tional government, and politics. They should know that constitutional governments can take different forms, ond they should 

understand the fundamental principles of American constitutional government and politics, including functions of political parties 
and other organizations. They should understood both rights and responsibilities in a demoaatk society, and they should 
recognize the volue of political portidpotion. They should be fomitior with internationol issues that affect the United States. 

Twelfthgrade students performing at the basic level should hove on understanding of what is meant by civil society, constitutional government, and politiG. 

They should know that constitutional governments can take different forms, and they should understand the fundamental principles of American constitutional 
government. These students should be able to explain ways that political parties, interest groups, and the media contribute to elections, and they should be able 
to point out sources of information about public policy issues. They should understand that both power and rights must be limited in a free society. They should 
be able to Identify those traits that make people responsible citizens, and they should be able to describe forms of political participation available in a democracy 
and recognize reasons that such participation is important. These students should be able to provide simple intepetations of non-text*based information, like 
mops, charts, tobies, graphs, and cartoons. Finally, they should be familiar with international issues that affect the United States. 

ProOdent Twelfth-grade students performing ot the proficient level should have a good understanding of how constitutions can limit the 

(174) power of government ond support the rule of low. They should be able to desaibe similorities and differences among constitu- 
tionol systems of government, and they should be able to explain fundomental Americon demoaatk values, their oppileations, 
and their contribution to expanding politkal participation. They should understood the structure of Amerkan government and be 
oUe to evoluote octhrities of political parties, interest groups, ond medio in public affoirs. They should be oble to explain the 
importonce of politkol portidpotion, pubik service, and politkal leodership. They should be oble to desaibe major elements of 
American foreign polky ond the performonce of major internationol organizations. 

Twelftb^rade students performing at the proficient level should hove a good understanding of how constitutions can limit the power of government and support 
the rule of low. They should be able to distinguish between parliamentary systems of government and those based on separate and shared powers, and they 
should be able to describe the structure and functions of American government. These students should be able to identify issues in which fundamental democratic 
values and principles are in conflict — liberty and equality, individual rights and the common good, and majority rule and minority rights, for example — and 
they should be able to toke and defend positions on these issues. They should be able to evoluote ways thot law protects individual rights and promotes the 
common good in American society. They should understood how the application of fundamental principles of American constitutional democracy has expanded 
participation in public life, and they should be able to explain how citizens can work individually and collectively to monitor and influence public policy. These 
students should understond the Importance and means of participation in political life at the national, state, and local levels. They should be able to evaluate 
contributions mode by political parties, interest groups, and the media ta the development of public policy, and they should be able to explain how public service 
and political leadership contribute to American democracy. They should understand how American foreign policy is made and carried out, ond they should be able 
to evaluate the performance of major international organizations. Finally, these students should be able to discuss reasons for and consequences of conflicts that 
arise when internotional disputes cannot be resolved peacefully. 

AJfOacad Twelfth-grade students performing at the advanced level should have a thorough and mature understanding of the strengths 

(294) and weaknesses of various forms of consHtutionol demoaocy. They should be oble to exploin fully the structure of Amerkan 
government ond the politicol process. They should understand differences between Americon ideals and realities, and they 
should be able to explam past ond present responses to those differences. They should understand why civk dispositions ond 
individual and coDective politicol actions sustoin demoaocy. They should be oble to explam objectives and consequences of 
Americon foreign policy. 

Twelfth^rode students performing at the advanced level should have a thorough and mature understanding of the strengths and weaknesses of various forms of 
constitutional democracy. They should be able to discuss advantages and disadvantages of confederal, federal, and unitary systems of government, os well as 
strengths and weaknesses of porliamentary systems of government when compared with those based on separate and shared powers. These students should be 
able to explain how the structure of American government and the nation's social ond political cultures serve one another. They should know which level and 
agency of government to contact to express their opinions or influence public policy. They should be able to explain and evaluate past and present individual and 
collective political actions aimed at narrowing the gap between American ideals and notional reality. They should understand how elections help determine public 
policies, and they should be able to evoluote public policy issues in which fundamental values and principles are in conflict — liberty and equality, individual 
rights and the common good, and majority rule and minority rights, for example. These students should be able to evaluate the validity and emotional appeal of 
post and present political communicotion. They should be able to explain how civic dispositions such as civility, tolerance, and respect for law are important for 
preserving democracy, and they should be able to evaluate the many forms of participation in public affairs. Finally, they should be able to explain how American 
foreign policy is made and carried out and ta evaluate Its consequences. 



SOURCE; National Assessment Governing Board. Policy adopted May 15, 1999 at quarterly meeting. Washington, DC. 
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The NAEP legislation requires that achievement levels be “used 
on a developmental basis until the Commissioner of Education Statistics 
determines ... that such levels are reasonable, valid, and informative to the 
public.” A discussion of the developmental status of achievement levels may 
be found in the Introduction (pages 12-14). 

The percentages of students at grades 4, 8, and 12 who performed at or 
above each of the achievement levels are presented in Table 1.2. In reading 
Table 1.2, keep in mind that the achievement levels are cumulative. That is, 
included among students who are considered to be at or above Basic are those 
who also have achieved the Proficient and Advanced levels of performance, and 
included among students who are considered to be at or above Proficient are 
those who have attained the Advanced level of performance. For example. 

Table 1 .2 shows that the percentage of fourth-grade students at or above Basic 
is 69 percent. The 69 percent includes not only students within the Basic level 
range, but also those whose performance was within the Proficient and the 
Advanced ranges. 



Table 1.2 



Percentage of students at or above the civics achievement levels 
for the nation: 1998 



1 Nation 




At ar 


Below 


above At or above 


Basic 


_ Basic Proficient Advnnfpd 


- L.1ZI] 


Grade 4 I 


31 


69 23 2 


. "Tj 


Grade 8 I 


30 


70 22 2 


, _^:ii 


Grade 12 | 


35 


65 26 4 



SOURCE: Nolionol Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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As displayed in Table 1 .2, about two-thirds of the students in each grade 
(69, 70 and 65 percent in grades 4, 8, and 12, respectively) were at or above the 
Basic level in civics performance in 1998. Performance at the Proficient level — 
the achievement level identified by the NAGB as the level that all students 
should reach — was achieved by 23 percent of students at grade 4, 22 percent 
of students at grade 8, and 26 percent of students at grade 12. The highest level 
of performance. Advanced, was attained by 2 percent of students in grades 4 
and 8 and 4 percent of students in grade 12. 

Figure 1 .4 also shows achievement level results. The percentages in this 
figure differ from those in the previous table, because they are not cumulative 
— they represent the percentage of students within each achievement level 
range, rather than the percentage of students at or above each level. About one 
in three students at each grade level (31, 30, and 35 percent in grades 4, 8, and 
12, respectively) scored in the “below Basic” level. At grades 4 and 8, almost 
half of the students scored in the Basic achievement level, while at grade 12, 39 
percent of students were in the Basic achievement level range. 




Percentage of students within each civics achievement level range 
for the nation: 1998 
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Percentage Percentage Percentage Percentage 

below Basic at Basic at PraBcient at Advanced 



Grade 4 
Grade 8 
Grade 12 



Percentage of students at each achievement level 






1 31 1 46 






, 


1 30“1 48 


21 






1 35 1 39 


22 i 


-J 1 1 1 1 


I 1 1 1 1 



100 80 60 40 20 0 20 40 60 80 100 

Percentage at or bebw Bask Percentage at or above Profident 



NOTE: Percentages may not add to 100, or to the exoct percentages at or above achievement levels, becouse of rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 

1 998 Civics Assessment. 
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Sample Questions and Student Responses 

The following pages present sample questions and student responses from the 
1998 NAEP civics assessment that exemplify student performance within each 
of the three achievement level ranges: Basic, Proficient, and Advanced. One 
sample question for each achievement level is presented for each of the three 
grades assessed. An examination of these sample questions and student 
responses provides some insight into the extent of students’ civics knowledge 
and understanding at each of the achievement levels. 

A combination of multiple-choice, short constructed-response, and 
extended constructed-response questions is included. For each question, the 
aspect of civics being assessed by the question is identified. The correct answer is 
indicated for multiple-choice questions by a star (★). For constructed-response 
questions, a summary of the scoring criteria used to rate students’ responses is 
provided. Actual student responses have been reproduced from assessment test 
booklets. The rating assigned to each response is also indicated. 

The tables included with each exemplary question provide two types of 
information about students’ performance on that question. First, the overall 
percentage of students who answered successfully is provided. This is the 
pefcentage of all students, regardless of achievement level performance, who 
were able to answer the question correctly. This overall percentage of students 
includes those who were below Basic, as well as students whose performance fell 
within the three achievement level ranges. Second, the percentage of students 
within each of the three achievement level ranges — Basic, Proficient, and 
Advanced — who answered successfully is provided. These percentages give 
some indication of how difficult the question was for students who performed 
at each achievement level. The scale score ranges that correspond to the 
achievement Ifvel ranges are provided for reference. 

Over 100 test questions released from the 1998 NAEP civics assessment 
are available for viewing on the NAEP website at http://www.nces.ed.gov/ 
nationsreportcard/itmrls/intro.shtml. In addition, the item-viewing feature 
of the website includes detailed scoring guides (rubrics) for the constructed 
response questions, sample student responses, and student performance data. 
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Grade 4 Basic Level - Sample Question and Response 



Scott wants to be a police officer when he grows up. He says the 
police get to wear fancy uniforms with badges, use handcuffs, 
and drive cars as fast as they want. What is wrong with Scott's 
ideas about why he wants to be a police officer? 

Me. W ^0 WWig onA h>e.Ccfast> 

V %V>rQ>a.Q. cM\e.rS. 

Think about the things police officers do in their work. What are 
two good reasons to be a pohce officer? 

A'xScvylioe people So \4\e^ com \ Warn ^ok\ 

/y\‘>s4ox/te£. 

9.) \p^u»eev\ f Co^le ^Vick. 4 ore 

rxi\A ^Kp ^probleA»>^> 



This construeted-response question is designed to measure fourth-grade 
students’ ability to make distinctions between power and authority. The 
response received a score of 3, or ‘‘Acceptable,” on a four-point scale in which 
a score of 4 was considered “Complete” and a score of 1 was considered ^ 
‘‘Unacceptabk^ of the response did not receive credit because 

ite meaning wa^^^^ both reasons for being a police officer 

were credited. 



Table 1 .3 I Grade 4: Functions and purpose of government 





Percentage "Acceptable" or better within achievement level ranges 


Overall percentage 


Bask 


Proficient 


Advanced 


"Acceptable" ar better 


136-176* 


177-214* 


215 and abave* 


67 


71 


87 


** * 



* NAEP civics scale range. 

*** Sample size is Insufficient to permit a reliable estimate. 

SOURCE; National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1 998 Civics Assessment. 
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Grade 4 Proficient Level - Sample Question 



11. Which of the following is the most 

important reason why the United States 
trades with other coxmtries? 

® People get a chance to travel. 

★ ® It helps people get the things they need. 

<D It helps us learn about other cultures. 

CE) We can learn other languages. 



This question is designed to measure whether fourth-grade students understand 
the concept of international trade. Options A, C, and D may all be by-products 
of trade with other coimtries, but option B is clearly- the “most important” 
reason for, trade. Fourth graders at the Proficient level were likely to choose the 
correct response. 



Table 1.4 j Grade 4: Relationship of the U.S. to other nations 
I and to world affairs 





Percentage "Correct" within achievement level ranges 


Overall percentoge 


Basic 


Proficient 


Advanced 


"Correct" 


136-176* 


177-214* 


215 and above* 


■ 49 


49 


70 


*** 



* NAEP civics scale range. 

*** ^mple size is insufficient to, permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1 998 Civics Assessment. 
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Grade 4 Advanced Level - Sample Question 



Question 7 refers to the cartoon below. The word apathy in the 
cartoon means "not caring." 



-iHQliiiiatiiKiiK 




Ct^iyright 0 1991 Conunus in Saratoga Herald Joiunal. 

What is the message of the cartoon? 

★ CD Democracy could be in danger if people do not vote. 

CD People like to get all of their political ideas from television. 

CD People do not care whether they have the right to freedom 
of speech. 

C® ,It!isjhard‘;to be a candidate for President. 



In this question, students were asked to interpret a cartoon about the relationship 
between democracy and civic participation and to answer a multiple-choice 
question based on it. Students, particularly at the younger grades, tended to 
find political cartoons difficult to understand. In this case, the presence of a 
cartoon coupled with a sophisticated concept led to low percentages of students 
answering correctly. 



Table 1 *5 I Grade 4: Roles of U*5. citizens in American democracy 





Percentage "Correct" within achievement level ranges 


Overall percentage 


Bask 


ProBcienI 


Advanced 


"Correct" 


136-176* 


177-214* 


215 and above* 


26 


26 


35 


« « « 



* NAEP civics scale range. 

*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1998 Civics Assessment. 
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Grade 8 Basic Level - Sample Question 



Two countries both claim that an island in the Pacific Ocean 
belongs to them. The countries are preparing to go to war with 
each other over this issue. 

What is the United Nations able to do to help end the conflict? 

CB> Send weapons to both sides. 

CD Disarm the militaries of both countries. 

★ CD Arrange for diplomatic negotiations between the two 
countries. 

CD Force all other countries to stop trading with the two 
countries. 



Eighth graders at the Basic level were likely to choose the correct option to this 
multiple-choice question. It falls within the content category United States and 
its relationship to other countries and world affairs. Specifically, it is designed to 
measure students’ understanding of powers and limitations of the United 
Nations as an arbiter of international conflict. 



Table 1 .6 | Grade 8: Relationship of the U.S. to other nations 

and to world affairs 





Percentage "Correct" within achievement level ranges 


Overall percentage 


Basic 


Proficienf 


Advanced 


"Correcf 


134-177* 


178-212* 


213ondabove* 


77 


84 


94 


*** 



* NAEP civics scale range. 

*** Sample size is insufFicient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1 998 Civics Assessment. 
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Grade 8 Proficient Level - Sample Question 



This question refers to the passage helow: 

When two [people] come into [the Supreme] Court, one may 
say: "an act of Congress means this." The other may say it 
means the opposite. We [the Court] then say it means one of 
the two or something else in between. In that way we are 
making the law, aren't we? 

— Earl Warren, Chief Justice of the Supreme Court 

Some people are troubled by the role of the Court described by 
Chief Justice Warren. Which argument could they effectively 
use against it? 

★ ® It is dangerous to give nonelected officials such as judges so 
much power in the government. 

® The Supreme Court makes it too difficult for the federal 
government to exercise its power over the states. 

® Supreme Court judges are the members of society most 
capable of making decisions about social policy. 

® The main task of the Supreme Court is to rewrite the 
Constitution to respond to modern problems. 



This multiple-choice question is the second of a two-question set covering 
the general theme of the distribution and sharing of powers by the three 
branches of the federal government. Both questions refer to the quote from 
Chief Justice Warren. The first question asks students to identify that Warren 
was writingiabout judicid review. The question shown here probes more deeply 
sb ’th^f students can demonstrate an understanding of the conflicting views 
on the power exercised by the courts. Students found the question to be 
fairly difficult. 



Table 1 *7 I Grade 8: Foundations of the American political system 





-- '' 


r' ■ ■ 


■ ' ' 




Percentage "Corrett" wthin achievement level ranges 


r Overall percentage 


Banc 


ProBdenI 


Advanced 


"Correct" 


134-177* 


178-212* 


21 3 and above* 


31 


26 


56 


♦ ♦ ♦ 



* NAEP civics scale range. 

*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1998 Civics Assessment. 
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©irsidle 8 Ls'^^eD - SatraipBe ©lojiesfi'Soirii siinidl [®esp@oTise 



Give two specific examples of how the United States 
Constitution limits the power of the government. 



1} ^XHro O y ^d; c\ o.\ 



The constructed-response question shown here is a straightforward attempt 
to measure students’ understanding of the ways in which the United States 
Constitution limits the power of the government. The question was scored on 
a three-point scale. A “Complete” response had to provide two distinct ways in 
which the Constitution limits government. A “Partial” response had to give one 
way the Constitution limits government. An “Unacceptable” response did not 
describe any way's in which the Constitution limits government. A “Complete” 
score-could not be earned if the answer to one part of the question was a subset 
of the answer to the other part. For example, a response of “Bill of Rights” and 
“freedom of speech” would receive partial credit because freedom of speech is 
found within the Bill of Rights. The response above received a score of 
“Complete” because it provided two different and correct answers. 



Table T,8 



Grade 8: 





Percentoge "Complete" within ochievement level ranges 


Overall percentoge 


Basic 


ProHcienl 


Advanced 


"Complete" 


134-177* 


178-212* 


21 3 end obove* 


13 


10 


29 


*** 



* NAEP civics scale range. 

*** Sample size is Insufficient ta permit a rdiable estimate. 

SOURCE: Natlanal Center for Educatian Statistics, Natianal Assessment af Educatianal Progress 
(NAEP), 1 998 Civics Assessment. 
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This question refers to the statement below: 

The United States is not a fully democratic country. 

The framers of the Constitution created a system in 
which majorities — even large majorities or their 
representatives in Congress — do not have the right to do 
anything and everything they want. 

The framers of the Constitution wanted to limit the 
power of majorities in order to 

<S> encourage the growth of political parties 

CD ensure that state governments would remain weak 

CD enable the government to act quickly in times of crisis 

★ CD protect the rights of individuals and minorities 



This question is the second of a two-question set based on a short passage about 
the power of majorities in the United States. Although the passage is 
not essential to answering the question, it is intended to help stimulate student 
thinking. The question covers the idea, found under the Foundations of the 
American Political System portion of the Framework, that the Constitution 
legitimizes majority rule in certain key areas of decision-making, but limits 
the power of majorities in order to protect the rights of individuals. 



Table 1 .9 I Grade 1 2; Foundations of the American political system 





Percentage "Correct" within achievement level ranges 


Overall percentage 


Basic 


Proficient 


Advanced 


"Correct” 


139-173* 


17+-203* 


204 and above"' 


72 


78 


93 


97 



* NAEP civics scale range. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1 998 Civics Assessment. 
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This question refers to the passage below: 

"Absolute arbitrary power, or governing without settled 
laws, can neither of them be consistent with the ends of 
society and government." 

— John Locke 

List two ways the American system of government is designed 
to prevent "absolute arbitrary power" and "governing without 
settled laws." 

1) The sys^yyj of Checks qhc/ 

javei/SniS cenh-iio hraK^k af ^Ot^ry)Meiv^ 

frotA ge>-f/iA3 -foo ^o\a^Y^uI. 

2) TTie Aiyigtog/rtLftyvi ^ybcess ol//oia/s h 

j-»p aJAsA of alHre^ ia f/e hes^ 

heeol^ of 



This open-ended question is the second in a two-question set based on the 
quote from Locke. The quote does not refer to the United States in particular, 
but students are expected to explain how Locke’s idea is manifested in the 
United States constitutional system. 

The question was scored on a 3-point scale in which a “Complete” 
response correctly lists two ways that the American government is designed to 
prevent the problems mentioned by Locke, a “Partial” response lists one way, 
and an “Unacceptable” response does not give any ways. The response above 
received a score of “Complete” because both parts mention aspects of the 
United States constitutional system that are designed to prevent absolute 
arbitrary power and governing without settled laws. 



Table 1.10 I Grade 12: Functions and purpose of government 





Percentage "Compiete" within achievement level ranges 


Overall percentage 


Bask 


ProHdenl 


Advanced 


"Complete" 


139-173* 


174-203* 


204 and above^ 


25 


24 


51 


75 



* NAEP civics scale range. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1 998 Civics Assessment. 
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Grade 1 2 Advanced Level - Sample Question 



This question refers to the statement helow: 

The United States is not a fully democratic country. The 
framers of the Constitution created a system in which 
majorities — even large majorities or their representatives 
in Congress — do not have the right to do anything and 
everything they want. 



Which aspect of the American system of government shows one 
of the limits on the power of majorities discussed above? 

<35 The ability of Congress to override presidential vetoes 

★ CD The Supreme Court's power to overturn unconstitutional laws 

<D The right of Congress to impeach Presidents and federal 
judges 

The ability of people in many states to vote public initiatives 
into law 



This question was meant to measure smdents’ understanding of the 
constimtional limits on the power of majorities and to test smdents’ ability to 
interpret a quote. The question was paired with another multiple-choice 
question that asked why the framers of the Constimtion wanted to limit the 
power of majorities. 



Table 1 . 1 1 | Grade 1 2: Functions and purpose of governnient 





Percentage "Correct" within achievement level ranges 


Overall percentage 


Basic 


Proficient 


Advanced 


"Correct" 


139-173* 


174-203* 


204 and above* 


30 


20 


42 


85 



* NAEP civics scale range. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1998 Civics Assessment. 
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Gender 



The 1998 average civics scale scores for males and females are presented in 
Table 2.1. In this table and in all the tables in this chapter, the percentage 
of students in each subgroup (for example, the percentage of females in the 
sample) is presented alongside the average scale score. At grades 8 and 12, 
female students had higher average scale scores than their male peers. At grade 
4, the difference between males and females was not statistically significant. 
The results of the 1998 NAEP civics assessment differ from the results of 
previous NAEP surveys in the social studies areas, in which male students 
outperformed female students (particularly at the twelfth grade). ^ 




Average civics scale scores by gender: 1998 





Percentage 
af students 


Average 
scale score 


1 




Grade 4 


Male 


52 


149 


Female 


48 


151 




Grade 8 I 


Male 


51 


148 


Female 


49 


152 


i 


Grade 12 1 


Male 


48 


148 


Female 


52 


152 



NOTE: Percentages may nat add ta 100 due ta raunding. 

SOURCE: Natianal Center far Educatian Statistics, 
Natlanal Assessment af Educatianal Pragress (NAEP), 

1 998 Civics Assessment. 



^ On the 1988 NAEP civics assessment, for example, male students outperformed female students at the 
twelfth grade. Before interpreting this as a change in gender patterns of performance, however, readers 
should note that the 1998 assessment was different from its 1988 predecessor. The 1998 assessment used 
both multiple -choice and open-ended questions throughout, while the 1988 assessment contained only one 
open-ended exercise. For details, see Appendix A. The results in civics also differ from those in the 1994 
NAEP Geography assessment (on which males outperformed females at all three grades), and the 1994 
NAEP U.S. History assessment (on which males outperformed females at grade 12). 
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Average Civics Scale Score 
Results for Selected Subgroups 



Overview 

This chapter presents average civics scale score results for various subgroups 
of students. The findings are summarized on the National Assessment of 
Educational Progress (NAEP) civics scales, which have ranges of 0 to 300 at 
each grade. The results from each grade are summarized on an independent 
scale; cross-grade comparisons are not possible. 

An examination of the“ average scores of subgroups provides insight into 
how general patterns bf civics performance are related to certain background 
characteristics. Results are reported by gender, race/ethnicity, parental 
education, region, type of location, eligibility for the Free/Reduced-Price 
School Lunch Program, and type of school. Achievement level results for the 
same subgroups are’ presented in Chapter 3. 

The only subgroup differences discussed in this report are those that pass 
tests of statistical significance. These tests consider both the magnitude of 
the difference betwejcn the subgroups’ average scores or percentages and the 
standard errors of the statistics.^ Readers are reminded, however, that findings 
of statistical significance are not judgments about the size or educational 
importance of the differences discussed. 

In interpreting subgroup results, readers are reminded that differences 
in performance reflect a range of socioeconomic and educational factors not 
discussed in this report or addressed by the NAEP program. Important issues 
such as opportunities to learn and sociocultural environmental factors must be 
considered in interpreting differences between groups.^ Therefore, readers 
should avoid making simple or causal inferences based on these data. 



* See Appendix A for a more detailed discussion of statistical significance testing procedures. 

^ Stevens, F. (1993). Opportunity to learn: Issues of equity for poor and minority students. Washington, DC: 
National Center for Education Statistics. 
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Race/Ethnicily 

As part of the background questionnaire that was administered with the NAEP 
1998 civics assessment, students were asked to indicate the raciai/ethnic 
subgroup that best described them. The mutudJy exclusive response options 
were; White, Black, Hispanic, Asian/Pacific Islander, and American Indian 
(including Alaskan Native). 

The 1998 average civics scale scores for students in racial/ethnic subgroups 
are presented in Table 2.2. As in previous assessments in a variety of academic 
subjects, differences in civics performance among racial/ethnic groups were 
evident at aU three grades. At grade 4, White students had higher average 
scale scores than children in each of the other ethnic groups. Asian/Pacific 
Islander students, while scoring at a lower level than their White counterparts, 
had higher average scale scores than Black, Hispanic, and American Indian 
students. Finally, Hispanic students scored, on average, at a lower level than 
Black and American Indian participants. 

At grade 8, White students had higher average scale scores than their 
Black, Hispanic, and American Indian counterparts. Black and Asian/Pacific 
Islander students both scored, on average, at a higher level than Hispamic eighth 
graders. While the gaps between eighth-grade ./^ian/Pacific Islander students 
and their Black and American Indian counterparts appear substantial, they are 
not statistically significant."* 

Among high school seniors. White and Asian/Pacific Islander students 
scored, on average, at a higher level than Black, Hispanic, and American Indian 
students. There were no other statistically significant differences among ethnic 
groups at this grade. 



^ While the average scale scores for the Asian/Pacific Islander group are on the order of 20 points higher 
than those of the other groups, the differences are not statistically significant because the estimates 
of standard error used in calculating the significance tests are not reliably determined. Standard error 
estimate of low reliability can result when a large proportion of the data that make up a particular subgroup 
(in this case, Asian/Pacific Islander students) are clustered in relatively few schools. This occurs with the 
Aaan/Pacific Islander group because certain geographic areas, and individual schools, in the U.S. have high 
percentages of Asian/Pacific Islander enrollment, but most do not. The reliability of a standard error 
estimate is quantified by what are referred to as degrees of freedom. See Appendix A (page 128) for the 
formulae used in estimating degrees of freedom for NAEP significance tests, and a discussion of the impact 
of clustering effects on the estimation of standard errors. 
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Average civics scale scores by race/ethnicity: 1998 



Percentage Average 

of students scale score 




White 


67 




159 


Black 


15 




132 


Hispanic 


13 




126 


Asian/Pacific Islander 


2 




153 


American Indian 


2 




137 




White 


67 




159 


Black 


15 




133 


Hispanic 


14 




127 


Asian/Pacific Islander 


3 




153 


American Indian 


1 




134 


H 




Grade 12 




White 


6? 




158 


Black 


14 




131 


Hispanic 


12 




130 


Asian/Pacific Islander 


4 




151 


American Indian 


1 




129 



NOTE: Percentages may not add to 1 00 due to rounding. 

SOURCE: Notional Center for Education Statistics, 
Notional Assessment of Educational Progress (NAEP), 
1998 Civics Assessment. 
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Parents' Highest Level of Education 

Students were asked to indicate the highest level of education completed by 
each parent. Four levels of parental education were identified: did not finish 
high school, graduated from high school, had some education after high school, 
and graduated from college. The option “I don’t know” was also available. For 
this analysis, the highest education level reported for either parent was used to 
categorize students. 

Previous NAEP assessments in all subject areas have found that student- 
reported levels of parental education exhibit a positive relationship with student 
performance. However, some research has questioned the accuracy of student- 
reported data; therefore, caution should be used in interpreting the findings.^ 

In addition, it should be noted that 10 percent of fourth graders, 3 percent of 
eighth graders, and 1 percent of twelfth graders reported not knovwng the 
education level of their parents. 

The results for all levels of student-reported parental education are given in 
Table 2.3. On the grade 4 civics assessment, the relationship between student 
reports of parental education and scale scores appeared less pronounced than 
had been the case on other NAEP surveys;* Students at this grade who reported 
that neither parent had graduated from high school performed, on average, at 
lower levels than students who reported higher levels of parental education. 

At grades 8 and 12 the relationship between student reports of parental 
education and civics performance appeared stronger than was the case at 
grade 4. In almost all cases, increases in reported levels of parental education 
were associated with increases in scale scores. The only exception to this pattern 
occmred at grade 8, where there was no statistically significant difference 
between the average performance of students who reported that at least one 
of their parents was a high school graduate and those who reported some post- 
high school parental education. 



^ Looker, E.D. (1989). Accuracy of proxy reports of parental status charaaeristics. Sociology of Educationy 
62(4), 257-279. 

^ The 1998 NAEP civics assessment used a different set of questions than past NAEP assessments to 
determine parents’ highest level of education. Consequently, patterns of relationships between this 
background variable and scale scores may differ slightly from past results. 
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Average civics scale scores by parents’ highest level of education- 
1998 



Graduated from college 
Some education after high school 
Graduated from high school 
Did not finish high school 
I don't know. 



Graduated from college 


49 


160 


Some education after high school 


27 


143 


Graduated from high school 


16 


144 


Did not finish high school 


5 


123 


1 don't know. 


3 


123 






Graduated from college 


52 


160 


Some education after high school 


27 


145 


Graduated from high school 


14 


140 


Did not finish high school 


6 


124 


1 don't know. 


1 


102 



NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 

1 998 Civics Assessment. 



Percentage 
of students 


Average 
scale score 




58 


153 


17 


150 


12 


153 


3 


124 


10 


139 
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In addition to providing results for the nation, NAEP assessments traditionally 
present results for the four regions of the country: Northeast, Southeast, Central, 
and West. (The composition of the regions is described in Appendix A.) Table 2.4 
presents regional results for all three grades. 

Comparisons of scale scores among the regions show differing performances 
across the country. At grades 4 and 8, students in the Northeast and Central 
regions had higher average scale scores than their peers in the Southeast and 
West. In addition, fourth graders in the Southeast had higher average scores 
than their counterparts in the West. 

At grade 12, students in the Central region had higher scores than students 
in the other regions. Also, students in the Northeast performed better than 
those in the Southeast. There were no other statistically significant differences 
at this grade. 




Average civics scale scores by region: 1998 



Percentage Average 

of students scale score 







mi 


Northeast 


23 


156 


Southeast 


25 


145 


Central 


24 


159 


West 


27 


142 




1 . GrajEiie S ' 


, ... .1 


Northeast 


21 


155 


Southeast 


25 


143 


Central 


25 


156 


West 


29 


146 




1 Gfcicle Vi 


'' -'ll 


Northeast 


23 


151 


Southeast 


23 


145 


Central 


25 


157 


West 


29 


148 




NOTE: Percentages may not add to 100 due to rounding. 
SOURCE: National Center far Education Statistics, 



National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment 
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Each participating school in the NAEP 1998 civics assessment was classified 
according to its type of location. The three categories of location used in this 
report — central city, urban fnnge/large town, and rural/small town — are 
based on Census Bureau definitions. These classifications are strictly geographic 
(that is, the classifications are made based on the population size and density of 
a given location); economic characteristics are not taken into account in these 
categorizations. (The type of location classifications are described in Appendix A.) 
Table 2.5 presents scale score results for all three grades by type of location. 

Comparisons of performance in 1998 showed that fourth- and eighth-grade 
students in central cities had lower average scale scores than their peers in the 
other locations. At grade 8, students in rural locations had lower average scale 
scores than those in urban fi-inge/large town locations. At grade 12, there were 
no statistically significant differences among groups defined by type of location. 




Average civics scale scores by type of location: 1998 





Percentage 


Average 






of students 


scale score 




i 


Grade 4 


• — 




Central city 


35 


144 




Urban fringe/large town 


35 


154 




Rural/small town 


30 


152 






Central cily 


33 


145 




Urban fringe/large town 


40 


155 




Rural/small town 


28 


149 






Central cily 


32 


149 




Urban fringe/large town 


39 


152 




Rural/small town 


30 


149 





NOTE: Percentoges moy nof odd to 100 due to rounding. 

SOURCE: NoHonol Center for Educotion Stotistics, 
Notionol Assessment of Educotionol Progress (NAEP), 
1998 Civics Assessment. 
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iligibilily for the Free/Reduced>Pnce 

School Lunch Program 

The Free/Reduced-Price School Lunch component of the National School 
Lunch Program, offered through the U.S. Department of Agriculture, is 
designed to ensure that children near or below the poverty line receive nourishing 
meals/ The program is available to pubhc schools, nonprofit private schools, 
and residential child-care institutions. Ehgibility for free or reduced-price meals 
is determined through the USDA’s Income Ehgibility Guidelines and is 
included in this report as an indicator of poverty. These data are reported to 
NAEP by the individual schools. 




Average civics scale scores by Free/Reduced-Price School Lunch 



Program ehgibihty: 1998_ 





Percentage 
of students 




Average 
scale score 






Grade 4 




Eligible 


33 




132 


Not eligible 


54 




160 


Information not available * 


13 




154 






Grade 8 




Eligible 


26 




131 


Not eligible 


56 




157 


Information not available * 


18 




156 






Grade 12 




Eligible 


14 




130 


Not eligible 


66 




153 


Information not available * 


20 




153 



* As in other NAEP assessments, performance of students in 
the "information not available" category is similar to those 
in the "Not eligible" category. 

NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: National Center for Education Statistics, 

National Assessment of Educational Progress (NAEP), 

1 998 Civics Assessment. 



^ U.S. General Services Administration. (1995). Catalog of federal domestic assistance, Washington, DC: 
Executive Office of the President, Office of Management and Budget. 
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The results for students who took the NAEP civics assessment by eligibihty 
for the Free/Reduced-Price School Lunch Program are given in Table 2.6. At 
least 33 percent of students at grade 4, 26 percent of students at grade 8, and 
14 percent of students at grade 12 were eligible for the program.* At all three 
grades, students who were eligible for the Free/Reduced- Price School Lunch 
Program had lower average scale scores than students who were not eligible, or 
where the information was not available. 

Type of School 

Previous NAEP assessments and other survey research on educational 
achievement have found significant differences in the performance of students 
attending pubfic schools and those attending nonpublic schools.^ However, the 
reader is cautioned against using NAEP results to make simpfistic inferences 
about the relative effectiveness of public and nonpubfic schools. Average 
performance differences between the two types of schools may be related in part 
to socioeconomic and sociological factors. For example, some research points to 
instructional and policy differences between the two types of schools to explain 
the higher average scores of nonpubfic school students, while other studies 
have suggested that student selection and parental involvement are more 
significant contributors to performance differences." 

Average civics scale scores by type of school are presented in Table 2.7. 
Schools were clarified as either pubfic or nonpubfic; nonpubfic schools were 
then further divided into Catholic or other nonpubfic schools. At all three 
grades, students in nonpubfic schools had higher average scale scores than 
their peers in pubfic schools. Examining this difference more closely shows 
that students in Catholic schools had higher average scale scores than students 
in pubfic schools at aU three grades. Students in other nonpubfic schools 
outperformed those in pubfic schools at grades 4 and 12. The difference at 
grade 8, while apparently large, was not statistically significant.*^ 



* Information on eligibility was not provided for 13, 18, and 20 percent of students at grades 4, 8, and 12, 
respectively. Some of those students may also have been eligible for free/reduced-price lunches. 

^ Applebee, A., Langer, J.A., Mullis, I.V.S., Latham, A.S., & Gentile, C. (1994). NAEP 1992 writing report 
card. Washington, DC: National Center for Education Statistics. 

Campbell, J.R., Voelkl, K.E., & Donahue, P.L. (1997). NAEP 1996 trends in academic progress. 
Washington, DC: National Center for Education Statistics. 

Coleman, J., Hoffer, T., & Kilgore, S. (1982). Cognitive outcomes in public and private schools. Sociably 
of Education, 55, 65-76. 

Alexander, K.L. & Pallas, A.M. (1983). Private schools and public policy: New evidence on cognitive 
achievement in public and private schools. Sociology of Education, 56, 170-82. 

While the average scale scores for the nonpublic schools group are on the order of 21 points higher than 
those of the public schools, the differences did not reach statistical significance because the estimates of 
standard error used in calculating the significance tests are not reliably determined. Standard error 
estimates of low reliability can result when a large proportion of the data that make up a particular 
subgroup (in this case, nonpublic school students) are clustered in relatively few schools, or mean scale 
scores for the group are very disparate between certain school groupings. The reliability of the standard 
error estimate is quantified by what are referred to as effective degrees of freedom. See Appendix A 
(page 128) for the formulae used in estimating degrees of freedom for NAEP significance tests. 
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Average civics scale scores by type of school: 1998 
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NOTE; Percentages may not add to 100 due to rounding. 

SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 

1 998 Civics Assessment. 
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Summary 

For the NAEP 1998 civics assessment, the following patterns of scale score 

results across subgroups of students were observed: 

► Gender: Female students had higher average scale scores than male students 
at grades 8 and 12. 

^ Race/ ethnicity: At grade 4, White students had higher average scale scores 
than their Asian/Pacific Islander peers, who in turn had higher average scale 
scores than their Black, Hispanic, and American Indian counterparts. At this 
grade, Hispanic students scored at lower average levels than members of 
other ethnic groups. At grades 8 and 12, White students had higher average 
scale scores than Black, Hispanic, and American Indian students. At grade 8, 
Black and Asian/Pacific Islander students scored at higher levels than their 
Hispanic counterparts. At grade 12, Asian/Pacific Islander students performed 
at a higher level than their Black, Hispanic, and American Indian peers. 

^ Parental education: In general, higher levels of parental education were 
associated with higher levels of student performance, especially at the upper 
grades. For example, high school seniors who reported higher levels of 
parental education had higher;,average scores. This pattern was less pronounced 
at grade 4, where student reports of parental education are less reliable. 

^ Region: Students in the Northeast and Central regions had higher average 
scale scores than those in the Southeast and West at grades 4 and 8. In 
addition, at grade 4, students in the Southeast outperformed those in the 
West. At grade 12, students in the Central region outperformed their peers 
elsewhere, while seniors in the Northeast had higher average scores than 
those in the Southeast. 

^ Type of location: At grades 4 and 8, students from schools in urban fringe/ 
large town and rural/small town locations had higher average scale scores 
than their peers in central city schools. At grade 8, students from schools in 
urban fringe/large town locations outperformed their peers from schools in 
rural/small town locations. 

^ Free/Reduced-Price School Lunch Program eligibility: The NAEP 1998 
civics assessment collected information on this federally funded program, an 
indicator of poverty. Results indicated that, at all three grades, students who 
were eligible for the Free/Reduced-Price School Lunch Program had lower 
average scale scores than students who were not eligible. 

^ Type of school: At all three grade levels, students attending nonpublic 
schools had higher average scale scores than their counterparts attending 
public schools. 
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Civics Achievement Level 
Results for Selected Subgroups 

Overview 

In Chapter 2, average civics scale score results were presented for the selected 
student subgroups: gender, race/ethnicity, parental education, region, type of 
location, eligibility for the Free/Reduced-Price School Lunch Program, and 
type of school. This chapter presents achievement level results for these same 
subgroups. The three civics achievement levels — Basic^ Proficient^ and 
Advanced — defined in Chapter 1 are used to report these results. 

Gender 

The percentages of male and female students attaining the Basic^ Proficient^ and 
Advanced levels are given in Table 3.1 and Figure 3.1. Table 3.1 shows the 
percentage of students at or above each of the achievement levels. In reading 
Table 3.1, keep in mind that the levels are cumulative. That is, included among 
students who are considered to be at or above Basic are those who achieved the 
Proficient and Advanced levels of performance, and included among students 
who are considered to be at or above Proficient are those who attained the 
Advanced level of performance. Figure 3.1 shows achievement level results in 
terms of the percentages of students within each achievement level range. All the 
tables and figures in this chapter follow these same formats. 
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Different patterns of results were observed at each of the three grades. 

In Table 3.1, at grade 8, the percentage of students at or above the Basic level 
was higher among females than among males, while comparable percentages of 
both groups attained the Proficient and Advanced levels. At grade 12, a higher 
percentage of females than males were at or above the Basic level, but 
a higher percentage of males than females attained the Advanced level. At grade 
4, there were no statistically significant differences in the percentages 
of students attaining any of the three achievement levels. 

As shown in Figure 3.1, performance differences between males and females 
at grades 8 and 12 appeared most evident at the Basic level. At grade 8, 51 
percent of females were at the Basic level and 27 percent were below the Basic 
level. For males, the corresponding percentages were 45 at the Basic 
level and 33 below the Basic level. At grade 12, 42 percent of females were 
at the Basic level and 32 percent were below the Basic level. For males, the 
corresponding percentages were 35 at the Basic level and 38 below the 
Basic level. 




Percentage of students at or above achievement levels in civics 
by gender: 1998 
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SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1998 Civics Assessment. 




48 



63 



CIVICS REPORT CARD o CHAPTER 3 




Percentage of students within each achievement level range in 
civics by gender; 1998 



THE N/mON’S 




m: 



Male 

Female 




Percentage Percentage Percentage Percentage 

below 6o5fc at Basic at Proficient at Advanced 




1 00 80 60 40 20 0 20 40 60 80 1 00 



Percentoge ot or betow Bask Perceoti^e at or above ProHdent 




* Lj I ! I I ^ ^ ^ I 1 — 

100 80 60 40 20 0 20 40 60 80 100 

Perc«itime at or betow Bade Pertenti^e ot or obove PnBd&tt 




_i I ^ i 1 1 » 1 1— 

100 80 60 40 20 0 20 40 60 80 100 

Percentoge ot or bebw Basic Percentt^ ot or above PnBd&Jt 



NOTE: Percentages may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
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Achievement level results by racial/ethnic group are presented in Table 3.2 
and Figure 3.2. The percentages of students performing at or above the three 
acluevement levels are shown in Table 3.2. Differences among racial/ethnic 
groups were seen at all three grades. The general pattern of results was that the 
percentages of students at or above each of the achievement levels appeared 
greater among White and Asian/Pacific Islander students than among the other 
three racial/ethnic groups. In most instances, these apparent differences were 
also statistically significant.’ 

At all three grades, the percentages of White students at or above the 
Proficient level were higher than the corresponding percentages of Black, 
Hispanic, and American Indian students. Similarly, the percentages of Asian/ 
Pacific Islander students at or above the Proficient level also appear higher at 
all three grades than those of Black, Hispanic, and American Indian students. 
However, the apparent differences between Asian/Pacific Islander students and 
American Indian students at grades 4 and 12, and between Asian/Pacific 
Islander students and Black and Hispanic students at grade 12 are not 
statistically significant.^ 

With one exception, the percentages of White and Asian/Pacific Islander 
students at or above the Basic level are higher than the corresponding 
percentages of Black, Hispanic, and American Indian students. At grade 12, 
the apparent difference in percentages at or above the Basic level between 
Asian/Pacific Islander and American Indian students is not statistically 
significant. Also, White students outperformed their Asian/Pacific Islander 
counterparts in terms of percentages at or above the Basic achievement level 
at grade 4. 

The percentages of students within each achievement level range are 
displayed in Figure 3.2 At grades 4 and 8, considerable variability across racial/ 
ethnic groups was evident in the percentages in the below Basic^ Basic^ and 
Proficient categories. The percentages of fourth- and eighth-grade students 
at the Advanced level were relatively small in all groups, ranging from 0 to 3 
percent. Variability in achievement level percentages across racial/ethnic groups 
was also evident at grade 12. However, the percentages of twelfiJi-grade 

At dll three grades, the percentages of Black, tiispanic, and American Indian students at the Advunced level 
were quite snriali. Because of difficulties associated with obtaining accurate standard errors for such small 
percentages, it was often not possible to carry out significance tests involving Advanced level results for 
these groups. 

2 While the percentages at or above the Proficient level for the Asian/Pacific Islander group in Table 3.2 
appear substantially higher than the corresponding percentages for the Black, Hispanic and American Indian 
groups, the difierences did not reach statistical significance because the estimates of standard error used in 
calculating the significance tests are not reliably determined. Standard error estimates of low reliability can 
result when a large proportion of the data that make up a particular subgroup (in this case, Asian/Pacific 
Islander students) are clustered in relatively finv schools. The reliability of the standard error estimate is 
quanted by what are referred to as degrees of freedom. See Appendix A (page 128) for the formulae used 
in estimating degrees of freedom for NAEP significance tests. 
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Percentage of students at or above achievement levels in civics by 
race/ethnicity: 1998 
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SOURCE: National Center Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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Percentage of students within each achievement level range 
in civics by race/ethnicity: 1998 
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SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 

1 998 Civics Assessment. 
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students at the Basic level varied litde by racial/ethnic group, ranging from 
33 percent (among Black and Hispanic students) to 41 percent (among White 
students). Among the racial/ethnic groups at grade 12, between 1 percent and 
5 percent of students performed at the Advanced level, with 5 percent of White 
students and Asian/Pacific Islander students in the Advanced category. 

Parents' Highest Level of Education 

Table 3.3 and Figure 3.3 present achievement level results based on students’ 
reports of their parents’ highest level of education. The percentages of students 
performing at or above the three achievement levels are shown in Table 3.3. 

The percentages of students within each achievement level range are displayed 
in Figure 3.3. As shown in Table 3.3, there is some degree of association 
between student-reported parental education levels and student achievement 
at all three grades. However, the association is more clearly evident at the higher 
grade levels.^ 

At grade 4, small percentages of students were at the Advanced level, 
regardless of parental education level, and no statistically significant differences 
among the groups were evident. The percentages of students at or above the 
Proficient level- and at or above the Basic level were smaller among students who 
reported that neither parent graduated from high school than among students 
who chose the other parental education options. 

At grade 8, the percentages of students whose p^ents graduated from 
college that were at the Advanced level, and at or above the Proficient level were 
higher than the comparable percentages for the other parental education 
groups .‘‘The percentage of eighth -grade students at or above the Basic level was 
highest among students whose parents graduated from college and lowest 
among students whose parents did not graduate from high school. At grade 12, 
achievement level results showed a strong association with parental education. 
Groups with higher parental education levels had higher percentages of students 
at or above each of the achievement levels. 



^ The greater degree of association between parental education level and achievement may be due at least in 
part, to increased validity of student-reported parental education levels among older children. 

^ The percentage at the Advanced level of students whose parents did not graduate high school was almost 0. 
Because of difficulties associated with obtaining accurate standard errors for such a small percentage, it was 
not possible to carry out significance tests involving Advanced level results for this group. 
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Percentage of students at or above achievement levels in civics 
by parents’ highest level of education: 1998 
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SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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Percentage of students within each achievement level range 
in civics by parents’ highest level of education: 1998 
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Region of the Country 



In addition to results for the nation, NAEP achievement level results are 
provided for four regions of the country: Northeast, Southeast, Central, and 
West. The percentages of students performing at or above the three achievement 
levels by region are shown in Table 3.4. (The composition of the regions is 
described in Appendix A.) 

Several differences among regions were observed in the 1998 achievement 
level results. In each of the four regions, only modest percentages of students 
were at the Advanced level at all three grades. At grade 4, the percentage of 
students at the Advanced level was higher in the Central region than in the 
West. No other significant differences among regions were seen at any grade 
at the Advanced level. 

At all three grades, higher percentages of students in the Northeast and 
Central regions were at or above the Proficient level than in the Southeast. The 
percentages of students at or above the Proficient level in the Northeast and 
Central regions also appear higher than in the West region at all three grades. 
However, the apparent differences at grade 12 between these three regions are 
not statistically significant. 

In, terms of performance at or above the Basic level, results mirror those for 
the Proficient [cvd at grades 4 and 8. The Northeast and Central regions had 
higher percentages of students at or above Basic than did the Southeast and 
the West. At grade 12, the percentage of students at or above Basic was again 
higher in the Central region than in the Southeast and the West. However, the 
remaining differences among the regions with respect to percentages at or above 
the Basic level were not statistically significant. 

The percentages of students within each achievement level range by region 
are displayed in Figure 3.4. At grade 4, variability across regions was evident in 
the percentages in ^e below Basic, Basic, and Proficient categories. At grade 8, 
the percentage of students in the Basic level ranged from 46 percent (among 
students in the West) to 49 percent (among students in the Central region). At 
grade 12, the percentage of students in the Basic level ranged fi-om 37 percent 
(among students in the Southeast) to 41 percent (among students in the 
Central region). 
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Percentage of students at or above achievement levels in civics 
by region; 1998 
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SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 



THE NATION’S 
REPORT 
CARD 



m- 




CIVICS REPORT CARD © CHAPTERS 



72 




Tv* --.v-;''-" 






THE NATION’S 
REPORT fT^ 



Percentage of students within each achievement level range in 
civics by region: 1998 
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Type ©I Locqfiors 



Table 3.5 and Figure 3.5 present achievement level results for all three grades by 
type of location: central city, urban fringe/large town, and rural/small town. 

The percentages of students performing at or above the three achievement levels 
are shown in Table 3.5. Figure 3.5 displays the percentages of students within 
each achievement level range. (The type of location classifications are described 
in Appendix A.) 

Different patterns of results were seen at each grade level. At grade 12, there 
were no statistically significant differences among the type-of-location groups at 
any of the three achievement levels. It is worth noting that five percent of 
twelftii-grade students in both central city and urban fiinge/large town 
locations reached the Advanced level. At grade 8, the percentages of students at 
or above the Basic and Proficient levels were higher among students from urban 
fringe/large town locations than among students from central city or rural/ 
small town locations. At grade 4, the percentage of students at or above the 
Proficient level was higher among urban fringe/large town locations than 
among the students in central city locations. At or above the Basic level, both 
urban fiinge/large town and rural/small town locations had higher percentages 
of students than did central city locations. 




THE WmON'S 
REPORT 



Percentage of students at or above achievement levels in civics 
by type of location: 1998 



m- 



o 

ERIC 



1 

i 

i 


Below 

Basic 


At or 
above 
Basic 


At or above 
Proficient 


Advanced 




Grade 4 




Central city 


38 


62 


19 


1 


Urban fringe/large town 


27 


73 


26 


2 


Rural/small town 


28 


,72 


23 


1 






Grade 8 1 


Central city 


36 


64 


19 


1 


Urban fringe/lqrge town 


25 


75 


26 


2 


Rural/small town 


31 


69 


21 


1 






Grade 12 1 


Central city 


36 


64 


25 


5 


Urban fringe/large town 


33 


67 


28 


5 


Rural/small town 


36 


64 


25 


3 



SOURCE: National Center for EducoHon Statistics, 
National Assessment of Educational Progress (NAEP). 
1 998 Civics Assessment. 
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Percentage of students within each achievement level range 
in civics by type of location; 1998 
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SOURCE: Notional Center for Education Statistics, Notional Assessment of Educational Progress (NAEP), 
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Efigibility for the Free/Reduced-Price 
School Lunch Program 



Table 3.6 and Figure 3.6 present achievement level results for each grade by 
students’ eligibility for the Free/Reduced-Price School Lunch component of the 
National School Lunch Program. Eligibility for the Free/Reduced-Price School 
Lunch program is reported to NAEP by the individual schools. The percentages of 
students performing at or above the three achievement levels are shown in Table 
3.6. Figure 3.6 displays the percentages of students within each achievement level 
range by eligibility for the Free/Reduced-Price School Lunch Program. 

Across the three grades, higher performance was observed for students who 
were not eligible for the program than for those who were eligible, except for 
the Advanced level at grade 4, where there were too few students to make the 
comparison. For example, as seen in both Table 3.6 and Figure 3.6, among fourth 
graders who were eligible for the program in 1998, 49 percent were at 
or above the level, 9 percent were at or above the Proficient level, and less 
than 0.5 percent were at the Advanced level. By comparison, the corresponding 
percentages were 80 percent, 30 percent, and 2 percent for those students not 
eligible for the Free/Reduced- Price School Lunch Program. 




Percentage of students at or al)ove achievement levels in civics 
by Free/Reduced-Price School Lunch Program eligibility: 1998 
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^ Percentage is between 0.0 and 0,5. 

* Performance of students in this category is similar to those 
irs the ^Nct eligible* category. 

SOURCE: National Center for Education Statistics, National 
Assessment of Educational Progress (NAEP), 1 998 Civics Assessment. 
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Percentage of students within each achievement level range in civics 
by Free/Reduced-Price School Lunch Program eligibility: 1998 
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SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 

1 998 Civics Assessment. 
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Type of School 

The percentages of public and nonpublic school students at all three grades who 
were at or above each of the achievement levels is shown in Table 3.7. Shown in 
Figure 3.7 are the percentages of students within each achievement level range 
by type of school. 

At all three grades, the percentages of students at or above each of the three 
achievement levels — at or above SusiCj at or above PToficicnt., and at Advnnccd 
— were higher among nonpublic school students than among public school 
students. There were no significant differences at any of the three grades 
between Catholic and other nonpublic schools in the percentages at or above 
each of the three achievement levels. Seven percent of students in nonpublic 
schools in grade 12 reached the Advanced level. 



Table 3.7 






Percentage of students at or above achievement levels in civics 
by type of school: 1998 
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SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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Figure 3.7 



Percentage of students within each achievement level range in 
civics by type of school: 1998 
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NOTE: Percentages may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 

1998 Civics Assessment. 
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Summary 

This chapter presented achievement level results for selected subgroups of 
fourth, eighth, and twelfth graders in the 1998 civics assessment. The major 
findings reported in this chapter are summarized below. 

^ Some gender differences in civics achievement were found at grades 8 and 
12. At both grades, the percentages of students at or above the Basic level 
were higher among females than among males. At grade 12, however, the 
percentage of students at the Advanced level was higher among males than 
females. 

► At all three grades, the percentages of White students at or above the 
Proficient achievement level were higher than those of Black students, 
Hispanic students, and American Indian students. A higher percentage of 
White students than Black or Hispanic students reached the Advanced level 
at grade 12. At grade 8, a higher percentage of Asian/Pacific Islander 
students than Black, Hispanic, or American Indian students were at or above 
Proficient level. A higher percentage of Asian/Pacific Islander students also 
reached the Proficient level at grade 4 than did their Black and Hispanic 
counterparts. 

► There is some degree of association between student-reported parental 
education levels and student achievement at all three grades. At grade 4, the 
percentages of students at or above the Proficient level and at or above the 
Basic level were smaller among students who reported that neither parent 
graduated from high school than among students whose parents were in 
the other three parental education groups. At grade 8, the percentages of 
students whose parents graduated from college that were at the Advanced 
level, and at or above the Proficient level, were higher than the comparable 
percentages for students selecting the other parental education groups. 

The percentage of eighth-grade students at or above the Basic level was 
highest among students whose parents graduated from college and lowest 
among students whose parents did not graduate from high school. At grade 
12, achievement level results showed a strong association with parental 
education. Groups with higher parental education levels had higher 
percentages of students at or above each of the achievement levels. 
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^ Several differences among regions were observed in the 1998 achievement 
level results. At grade 4, the percentage of students at the Advanced level 
was higher in the Central region than in the West. At all three grades, higher 
percentages of students in the Northeast and Central regions were 
at or above the Proficient level than in the Southeast. At grades 4 and 8, 
the Northeast and Central regions had higher percentages of students at or 
above Basic than did the Southeast and West. At grade 12, the percentage of 
students at or above Basic was higher in the Central region than in the 
Southeast and West. 

^ At grade 12, there were no statistically significant differences among the 
type-of-location groups at any of the three achievement levels. Five percent 
of grade-12 students from central cities and urban fnnge/large towns were 
at the Advanced level. At grade 8, the percentages of students at or above 
the Basic and Proficient levels were higher among students from urban 
fnnge/large town locations than among students from central city or rural/ 
small town locations. At grade 4, the percentage of students at or above the 
Proficient \cwc\ was higher among urban fnnge/large town locations than 
among the students in central city locations. At the Basic level, both urban 
fnnge/large town and rural/small town locations had higher percentages 
of students than did central city locations. 

► At all three grades, students who were not eligible for the Free/Reduced- 
Price School Lunch Program had higher levels of achievement than students 
who were eligible for that program. 

► At all three grades, the percentages at or above each of the achievement 
levels were higher among students attending nonpublic schools than among 
their peers attending public schools. 
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Contexts for Learning Civics: 
School/Teacher Policies 
and Practices 




Overview 

As noted in the Introduction, the learning of civics topics in the classroom takes 
place under such diverse course names as, for example. Civics, American 
Government, American History, and Social Studies. Civics learning can also take 
diverse forms both inside and outside the classroom. Many teachers use trips to 
governmental sites or museums, multimedia presentations, as well as a variety of 
active teaching strategies (such as group projects, mock trials, writing letters on 
civic topics, etc.) to enhance civics learning. Students may also learn civics from 
television, film documentaries, extraciuricular books, newspapers and magazines, 
or their contacts with community organizations. This chapter and the one that 
follows it examine some of the contexts in which civics learning occurs. This 
chapter focuses in particular on fourth- and eighth-grade civics teachers, 
including their educational backgrounds, their perception of their preparation 
to teach civics, their satisfaction with available instructional resources, and their 
use of educational technology. As in some other NAEP assessments, parallel 
information is not available for twelfth grade teachers. These teachers were not 
administered a background questionnaire because of the difficulty of Imking 
students to teachers in a civics-related class at this grade level. 



Who Teaches Civics? 

Because civics per se is not a college major, the teachers assigned to civics 
instruction in the foiu-th, eighth, and twelfth grades have a wide variety of 
academic backgrounds. As with other school subjects, there is also wide variation 
in the teachers’ level of academic degree and years of teaching experience. 

Teachers whose students were sampled for the NAEP civics assessment in 
grades 4 and 8 were asked to complete a questionnaire about their educational 
backgrounds, experience, and teaching methods. Because students rarely take 
courses in civics, the teachers responding to the questionnaire were most likely 
general teachers at grade 4 and social studies teachers at grade 8. In the NAEP 
analyses for this report, teacher questionnaire data and student performance data 
were matched so that the relationship between the two sets of variables could 
be examined. 
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Table 4T begins to answer the question ‘‘Who is teaching civics?” by 
displaying the percentage of students taught by teachers in several college 
degree categories. In addition, the table displays the mean scale score and the 
percentage of students at or above the Proficient level for each teacher degree 
category. About half of students at both grades 4 and 8 were taught civics 
by teachers who have a Bachelor’s degree. In the fourth grade, 43 percent 
of students were taught by teachers who had a Master’s degree or higher. In 
the eighth grade, 46 percent of students were taught by teachers who had a 
Master’s degree or higher. The fourth-grade students taught by Master’s-level 
teachers had higher average scores than fourth-grade students taught by teachers 
with Bachelor’s degrees. For eighth-grade students, there were no significant 
performance differences in the civics assessment by teacher’s level of education. 



Table 4.1 
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Percentage of students, average civics scale scores, and percentage at 
or above Proficient by teacher’s highest degree, grades 4 and 8: 1998 
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SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1 998 Civics Assessment. 
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Table 4.2 displays the percentage of students by their teachers’ under- 
graduate majors. At grade 4, over three-quarters of students (81 percent) were 
taught by teachers with education majors, while at grade 8, almost half the 
students (47 percent) had teachers with history or political science degrees. 
Fifteen percent of fourth grade and 17 percent of eighth-grade students were 
taught civics by teachers with other majors, including English or language arts. 
The remainder of the students were taught civics by teachers with various 
majors in education. The “education” major was created by aggregating the 
following undergraduate major/minor options from the teacher questionnaire; 
elementary education, secondary education, special education, bilingual 
education/FAT,, administration and supervision, curriculum and supervision, and 
counseling. For both fourth- and eighth-grade samples, the teacher s 
undergraduate major in college was not associated with student performance on 
the civics assessment. 




Percentage of students, average civics scale scores, and percentage 
at or above Proficient by teacher’s undergraduate major, grades 4 
and 8: 1998 
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Teacher preparation may also be reflected in the type of teaching certification 
held by the classroom teacher. Certification ranges from “advanced professional’’ 
types to “regular,” “temporary,” “provisional,” and “don’t have a certificate in 
my main assignment field.” Table 4.3 shows the percentages of students taught 
by teachers holding each of the certification types. At both grades 4 and 8, about 
three-quarters of the students were taught by teachers with regular certificates. 
Most of the remaining students were taught by teachers with an advanced 
professional certification. Fourth-grade students whose teachers had either 
advanced professional or regular teaching certification attained higher civics 
scores than students whose teachers had certification in the “temporary/ 
provisional” category. In the eighth grade, the small percentage (1 percent) 
of students taught by teachers in the “other” certification category outscored 
students whose teachers had advanced professional, regular, or probationary 
certification. The preponderance of students in that 1 -percent- sized category 
were enrolled in nonpublic schools. 



Table 4.3 
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Percentage of students, average civics scale scores, and percentage 
at or above Proficient by teachers’ type of certification, grades 4 
and 8: 1998 
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SOURCE: NoHonol Center for EducoHon SloHsfics, NoHonol Assessmenl of EducoHonol Progress (NAEP), 
1 998 Civics Assessment 
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The amount of prior teaching experience instructors bring to civics 
classrooms can be examined in Tables 4.4 and 4.5. Table 4.4 displays years 
of general teaching experience, while Table 4.5 indicates years of experience 
specifically teaching civics or government. In the fourth grade, only 11 percent 
of students were taught civics by teachers v^dth 2 years or less of general teaching 
experience, while 59 percent of fourth graders were taught by teachers with 
11 or more years of experience. A similar pattern is evident in grade 8, where 
10 percent of students had teachers in the “2 years or less” experience category, 
while 58 percent of students had teachers v^dth 11 or more years of general 
teaching experience. In the fourth grade, students with teachers from the least- 
experienced category had lower average scale scores than students in all the 
other teacher-experience categories. In contrast, at grade 8, the students’ civics 
average scale scores did not differ by category of teacher’s general experience. 
The data in Table 4.5 are specific to years of experience in teaching civics or 
government. While 11 percent or less of students had teachers with less than 
2 years of general teaching experience (Table 4.4), higher percentages of 
students in both grades (39 percent in grade 4 and 49 percent in grade 8) had 
teachers with 2 years or less experience teaching civics or government. However, 
in both grades 4 and 8, the students’ average scale scores did not differ by the 
amount of civics teaching experience the teacher brought to the classroom. 



Table 4.4 




Percentage of students, average civics scale scores, and percentage 
at or above Proficient by years of general teaching experience, 
grades 4 and 8: 1998 
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NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1998 Civics Assessment. 
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Table 4.5 




Percentage of students, average civics scale scores, and percentage 
at or above Proficient by years teaching government/civics, 
grades 4 and 8: 1998 
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SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1998 Civics Assessment. 



Classroom teachers continue to develop their instructional skills by taking 
additional courses, workshops, and professional development seminars in many 
topics. The civics teacher questionnaires included a question on the amount of 
time spent in the past 12 months in professional development seminars in social 
studies. The results from this question are displayed in Table 4.6. At the fourth 
grade, 42 percent of the students were taught by teachers who had attended no 
social studies workshops in the past year. At the eighth grade, 23 percent of 
students had civics teachers who had not attended a social studies workshop 
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during the past year. The relationship between student performance in civics 
and the extent of teacher attendance at professional development workshops 
shows a mixed pattern by grade level. In the fourth grade, students whose 
teachers were in the “less than 6 hours” of professional development category 
outscored students whose teachers indicated having taken 16-35 hours of 
workshops. In the eighth grade, by contrast, the students whose teachers had 
received 16-35 hours of workshops outscored students whose teachers indicated 
the “less than 6 hours” category. The amount of civics content in the social 
studies workshops could have varied widely and is one of a number of factors 
that may have contributed to the mixed results by grade level. 
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Percentage of students, average civics scale scores, and percentage at 
or above Proficient by amount of time teachers reported spending in 
professional development workshops in social studies during the last 
twelve months, grades 4 and 8: 1998 
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1 998 Civics Assessment. 
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How Well Prepared are Teachers? 

The civics teacher questionnaires included a series of questions on how well 
prepared the teachers felt in various areas related to civics instruction, either 
through college/university courses or workshops. Teachers could respond that 
they were well, moderately well, or not well prepared. Table 4.7 presents the 
results in terms of these response options. In the fourth grade, about half of the 
students were taught by teachers who considered themselves well prepared in 
social studies instruction and 60 percent of students were taught by teachers 
who rated themselves as well prepared in classroom climate and governance. 
However, less than 10 percent of the students were taught by teachers who felt 
they were well prepared in using the voluntary national standards for civics or in 
using computer software for social studies. The fourth-grade students who were 
taught by teachers who felt themselves well prepared in classroom climate/ 
governance outscored those students whose teachers did not consider 
themselves well prepared. 
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Percentage of students, average civics scale scores, and percentage 
at or above Proficient by teachers’ reported preparation, 
grades 4 and 8: 1998 
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NOTE: Percentages may nat add ta 100 due ta raunding. 

SOURCE: Natianal Center for Education Statistics, Notional Assessment of Educational 
Progress (NAEP), 1998 Civics Assessment. 
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In the eighth-grade sample, at least three-quarters of the students were 
taught by teachers who considered themselves well prepared in social studies 
instruction and classroom climate/governance. Similar to the fourth-grade 
pattern, low percentages of students were taught by teachers who felt they were 
well prepared in implementing the national standards for civics (11 percent) or 
in using computer software for social studies (13 percent). About half of the 
eighth-grade students had teachers who felt well prepared in using instructional 
materials in social studies. Those students outscored students whose teachers felt 
themselves not well prepared in using instructional materials. 

The results of additional teacher self-preparedness questions are presented 
in Table 4.8. More than four out of five students at both grades 4 and 8 were 
taught by teachers who felt themselves well prepared in classroom management 
and organization. At both the fourth and eighth-grade levels, the students who 
had these teachers outperformed students on the civics assessment whose 
teachers were in the “moderately well-prepared” category. About half of the 
students at both the fourth and eighth grades were taught by teachers who felt 
themselves well prepared in techniques of cooperative group instruction. 

Approximately one in four students (from 16 to 28 percent) across grades 4 
and 8 were taught by teachers who considered themselves well prepared in 
either the use of telecommunications or the use of computers. The eighth-grade 
students whose teachers considered themselves moderately well prepared in the 
use of telecommunications had higher civics assessment scale scores than those 
students whose teachers did not feel well prepared in this area. There is some 
disjunction between the percentages of students whose teachers felt well 
prepared in using computers in general (26-28 percent in Table 4.8) and the 
even lower percentages (6-13 percent in Table 4.7) whose teachers felt well 
prepared in using software for social studies. 
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Percentage of students, average civics scale scores, and percentage 
at or above Proficient by teachers’ reported preparedness to fulfill 
certain teaching-related tasks, grades 4 and 8: 1998 
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145 
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*** 


* ** 
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^ Percentage is between 0.0 ond 0.5. 

***Sample size is insufficient to permit o relioble estimote. 

NOTE: Percentoges moy not odd to 100 due to rounding. 

SOURCE: Notional Center for Education Statistics, Notiono! Assessment of Educotiono! 
Progress (NAEP), 1998 Civics Assessment. 
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How Much Class Time is Given to Civics? 



Teachers at both fourth and eighth grades were asked to indicate the length of 
class time per day they typically spent on social studies instruction. Table 4.9 
displays the results for the four options: less than 30 minutes; 30—44 minutes; 
45-60 minutes; and greater than 60 minutes. At the fourth grade, 63 percent 
of students received instruction in the 30-^ minute range. At the eighth grade, 
the 30—44 minute category also appeared to be the largest (48 percent), but the 
45-60 minute category was nearly as large (41 percent). Class time length was 
significandy related to civics scale score performance only in the fourth grade, 
where the students in both the 30-44 minute and 45-60 minute class rimes 
outscored their peers in the “less than 30 minutes” category. 



Table 4.9 




Percentage of students, average civics scale scores, and percentage 
at or above Proficient by class time per day spent on social studies 
instruction, grades 4 and 8: 1998 
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***5ample size is insufficient to permit a reliable estimate. 

NOTE: Percentages may not odd to 100 due to rounding. 

SOURCE: National Center for Education Statistics, Notional Assessment of Educational Progress (NAEP), 
1998 Civics Assessment. 
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Do Teachers Believe They Have Adequate Resources? 

Teachers responded to the question “Which of the following statements is true 
about how well your school system provides you with the instructional materials 
and other resources you need to teach your class?” Table 4.10 displays the 
pattern of results. For both grades 4 and 8, 52 percent of the students were 
taught by teachers who indicated they received most of the resources they 
needed. Almost a third of the students in both grades had teachers who said 
they only receive “some” of the resources they need. One percent of the 
students in either grade had teachers who said they received “none” of the 
resources they needed. In terms of student performance, in the eighth grade, 
students whose teachers received “most” and “all” of the resources they needed 
attained significantly higher civics scale scores than students in the “some” 
resources category. In the fourth grade, the pattern of results was similar, but 
the differences did not reach statistical significance. 



Table 4.10 
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Percentage of students, average civics scale scores, and percentage at or 
above Proficient by teachers’ reports on the availability of resources, 
grades 4 and 8: 1998 
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***Sample size is insufficient to permit a reliable estimate. 

NOTE: Percentages may nat add to 100 due to rounding. 

SOURCE: National Center far Education Statistics, Notianol Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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What is the Pattern of Technology Usage 
to Enhance Instruction? 

Effective implementation of technology to enhance instruction is a complex 
activity for today’s schools. It requires resource-intensive activity across at least 
three dimensions: availability of appropriate, accessible electronic equipment; 
teacher preparation and training, and availability of appropriate software and 
Internet access. Evaluation of each of these dimensions can be approached from 
the points of view of the school administration, the teachers, and the students. 
The questionnaire data available from NAEP can begin to sketch out a pattern 
of the use of technology in education across the nation, but the sketch should 
be interpreted with caution because of the limited nature of the information. 
Table 4.11 addresses the first dimension, the availability of computers as 
reported by school officials. The three questions displayed with “yes” or “no” 
answers in Table 4.11 were not mutually exclusive. It was possible, therefore, 
for a school to have answered “yes” both to “are computers available in all 
classrooms?” and to “are computers available in a separate computer lab 
available to classes?” Computer availability in all classrooms appeared more 
frequent in grade 4 (79 percent) than in grades 8 (45 percent) or 12 
(24 percent). 

Across the three grades, 36 to 41 percent of students were in schools that 
answered “yes” to whether computers were available to bring to classrooms 
when needed. In both grades 8 and 12, students whose schools answered “yes” 
to this question outscored students whose schools answered “no.” 
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Percentage of students, average civics scale scores, and percentage at or 
above Proficient by availability of computers, grades 4, 8 and 12: 1998 
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SOURCE: Notional Center for Educotion Statistics, Notionol Assessment of Educotionol Progress (NAEP|, 1998 Civics Assessment, 
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The second dimension, level of teacher preparation to use technology, has been 
touched upon in the previous sections of this chapter. In Table 4.8, it can be seen 
that most students at both the fourth- and eighth-grade levels had teachers who 
saw themselves as at least moderately well prepared to use both telecommuni- 
cations and computers. Sixty-four percent of fourth graders and 73 percent of 
eighth graders were taught by teachers who felt themselves well to moderately 
well prepared to use telecommunications. Even higher percentages of students 
(86 percent in the fourth grade and 85 percent in the eighth grade) had teachers 
who saw themselves as well to moderately well prepared to use computers. 
However, this level of perceived confidence in preparation did not extend to 
the use of software for social studies classes. In Table 4.7 it can be seen that only 
35 percent of fourth graders and 54 percent of eighth graders were taught by 
teachers who felt themselves at least moderately well to well prepared to use social 
studies software. The cause of this discrepancy for teachers between perceived 
preparedness with computers in general and yet less preparedness with using social 
studies software is not available in the questionnaire data. Some hypotheses might 
include possibly poor availability/quality of existing social studies software, lack of 
time for teachers to review existing software, or lack of time to learn and experiment 
with such software. 

The third dimension, the frequency of use of computers, software, and the 
Internet by teachers in the classroom, is addressed in Table 4.12. At grade 4, 

57 percent of students were taught by teachers who indicated that computers 
were available in the social studies classroom. In grade 8, the largest numbers 
of students had computer access only through laboratories or libraries. In both 
grades, students who had laboratory access to computers had higher mean scale 
scores than students whose teachers indicated that computer access was “not 
available.” Teachers’ self-perception of their own rather weak preparedness to use 
social studies instructional software was also exhibited in the frequency with which 
they use computer software in social studies class. As can be seen in Table 4.12, 

67 percent of fourth graders and 63 percent of eighth graders were in classes where 
the teacher indicated “never or hardly ever” using computer software in class. 
Fourth-grade students whose teachers indicated that they used computer software 
once to twice a month had higher average scale scores than students in the “never/ 
hardly ever” category. 

In recognition of the burgeoning importance of the Internet, teachers were 
asked about degree of Internet use. Almost three-quarters (73 percent) of fourth 
graders were in classes where teachers indicated that as part of social studies 
instruction they never or hardly ever had students access information through the 
Internet for use in the classroom. Internet usage appeared more common in the 
eighth grade, where 55 percent of students were in classes that never or hardly ever 
used it, while 35 percent used it once or twice a month. In both the fourth and 
eighth grades, students whose teachers had them access information through the 
Internet once or twice a month had higher scores than students whose teachers 
never had them use the Internet. 
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Table 4.12 



Percentage of students, average civics scale scores, and percentage 
at or above Proficient by use of computers and the Internet for social 
studies instruction, as reported by teachers, grades 4 and 8: 1998 
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^ Percentoge is between 0.0 and 0.5. 

“‘Sample size is insufficient to permit a reliable estimate. 

NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: National Center for Education Statistics, Notional Assessment of Educational Progress (NAEP|, 1998 Civics Assessment. 
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Summary 

This chapter examined primarily teacher-related variables to set a context for the 

learning of civics. The key findings about the demographic profile of teachers of 

students in fourth- and eighth-grade social studies classes include: 

► About half of students at both grades 4 and 8 were taught civics by teachers 
with a Bachelor’s degree. Forty-three percent of fourth graders and 46 
percent of eighth graders were taught by teachers who had Master’s or 
higher degrees. The fourth-grade students taught by Master’s level teachers 
had higher scores than fourth-grade students taught by teachers with 
Bachelor’s degrees. 

► For both fourth- and eighth-grade student samples, the teacher’s 
undergraduate major in college made no statistically significant difference 
in student performance on the civics assessment. 

► Fourth-grade students whose teachers had either advanced professional or 
regular teaching certification produced higher civics scores than students 
whose teachers had certification in the “temporary/provisional” category. 

^ Most students at both fourth and eighth grades generally had been taught by 
experienced social studies teachers. Only 1 1 percent of students at grade 4 and 
10 percent at grade 8 had teachers with 2 years or less of experience. In the 
fourth grade only, students with teachers from the least experienced category 
were outscored in the civics assessment by students in all the other teacher- 
experience categories. 

► Students at both fourth and eighth grades had teachers who rated 
themselves as well prepared in social studies instruction, classroom climate/ 
governance, classroom management, and the use of computers. However, 
teachers in both grades generally rated themselves as much less well 
prepared in using the voluntary national standards for civics/ government 
or in using computer software for social studies instruction. 

^ Teachers of about half of the students in both grades said that they received 
“most” of the resources they needed, while teachers of only 1 percent of 
students said that they received “none” of the resources they need. 
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^ Across all three grades, 36 to 41 percent of students were in schools that 
answered “yes” to whether computers were available to bring to classrooms 
when needed. In both grades 8 and 12, students whose schools had 
computers available outscored students whose schools did not have 
computers available. 

^ Sixty-seven percent of fourth graders and 63 percent of eighth graders were 
in classes where the teacher indicated “never or hardly ever” using computer 
software in class. Fourth-grade students whose teachers indicated that they 
used computer software once or twice a month had higher mean scale scores 
than students in the “never/hardly ever” category. 

^ Almost three-quarters (73 percent) of fourth graders were in classes where 
teachers indicated “never or hardly ever” having students access the Internet 
for social studies. Internet usage appeared more common in the eighth 
grade, where 55 percent of students were in classes that “never or hardly 
ever” used it, while 35 percent used it once or twice a month. In both the 
fourth and eighth grades, moderate frequency of Internet usage (once or 
twice a month) was associated with higher mean scale scores than the groups 
of students who had no Internet usage. 
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Contexts for Learning Civics: 
Classroom Practices and 
Student Variables 





Overview 

While the previous chapter focused on teachers, this chapter focuses on 
classroom activities and other contexts for learning civics topics. With the 
development of professional standards for teaching in various subjects and the 
implementation of the Goals 2000: Educate America Act‘ in the early 1990’s, 
policymakers, educators, researchers and the public have become increasingly 
interested in the type and quality of instruction in the nation s elementary and 
secondary schools. Of particular interest is the relationship between various 
classroom instructional practices and student learning. The NAEP data can 
begin to shed light on these issues. The complex interactions among teacher 
qualities, student ability levels, and patterns of implementation of instructional 
techniques are beyond the scope of this report card, but have been approached 
in other studies.^ 

The NAEP questionnaires administered to students and teachers provided 
the data for the analyses in this chapter. Although the analyses presented in this 
chapter can begin to address the nature of the relationship between teachers’ use 
of various instructional techniques and the civics scale scores of their students, 
sijcJi analyses provide only one perspective on this multidimensional question. 
Many additional studies would be needed to provide a more complete picture 
of the dynamics of elementary and secondary instruction in civics. 







• Executive Office of the President. (1990). National soals for education. Washington, D.C.: U.S^ 
Government Printing Office. Goals 2000: Educate America Act, H.R. 1804, 103d Cong., 2nd Sess. (1994). 
2 Henke R. R. Chen X 8c Goldman, G. (1999). What happens in clasirooms? Instructional practices in 
eUmentary and secondary schools, 1994 - 95 . (NCES PubUcadon No. 1999-348). Washington, DC; National 
Center for Education Statistics. 
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What Civics Topics do Students Study? 

Chapter 1 enumerated the range of topics from the civics framework that 
guided the development of the civics assessment. Do students encounter these 
topics in their social studies classes? The opportunity for school exposure to the 
topics found in the 1998 civics framework is important for good performance 
on the assessment. Table 5.1 presents the results for grade 4 for the question 
“Dining this school year, have you studied any of the following topics?” Topics 
studied by at least half the students included the student’s community, the 
President and leaders of the country, rules and laws of government, and rights 
and responsibilities of citizens. It should be noted that these percentage results 
may be somewhat understated because of two factors: first, students may have 
encountered these topics in the first, second, or third grades and yet (accurately) 
answered “no” to the question. Second, fairly large percentages of fourth 
graders (17 to 32 percent) chose the “I don’t know” option for these topics. 
Students who answered “yes” on the studied “your community” topic had 
higher average civics scale scores than did the students who answered “no.” 

In several topic areas in Table 5.1, grade-4 students who answered “no” to 
studying a topic in grade 4 had higher scores than students who answered 
“yes.” These counterintuitive results may stem either from students’ failure to 
understand the reporting categories or from their using knowledge they gained 
in earlier grades. 
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Percentage of students, average civics scale scores, and percentage at 
or above Proficient by type of content studied this year as reported by 
students, grade 4: 1998 
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NOTE: Percentages moy not odd to 100 due to rounding. 

SOURCE: NoHonol Center for Educotion Stotistics, Notionol Assessment of Educotionol Progress (NAEP|, 1998 Civics Assessment. 



More detailed questions on topics studied were asked of the eighth and 
twelfth graders. These are displayed in Table 5.2. Students in these grades 
generally indicated a high percentage of classroom coverage of the civics topics 
in the framework. Over 70 percent of students at both grades 8 and 12 
indicated that they had studied the U.S. Constitution and Congress in the 
current school year. Note that the same caution raised for the fourth-grade data 
applies to this question for grades 8 and 12 — students could have accurately 
answered “no” to the topics on the list because they studied them in grades 
other than their current year. In fact, on many of the topics, the twelfth graders 
who answered “no” had higher average civics scale scores than students who 
answered “yes.” This seemingly curious result may have its roots in the fact that 
the students had indeed covered the topics — but not in the current grade. 
However, more research is needed to examine this speculation. 

While about two-thirds or more of the eighth graders and twelfth graders 
indicated studying the first seven topics (with the exception of the President and 
cabinet) listed in Table 5.2, considerably lower percentages (generally less than 
half) indicated that they had studied other coimtries’ governments or 
international organizations such as the UN. 
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Table 5.2 
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NOTE: A small percentage of students responded "I don't know" to each of the topics. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1998 Civics Assessment. 




What Instructional Techniques do Civics 
Teachers Use? 

Civics teachers’ responses to questions about the frequency with which they 
use each of a selection of instructional activities were matched to their students’ 
civics assessment results for this analysis. The percentages of fourth-grade 
students receiving each instructional activity by “frequency of exposure” 
category are displayed in Table 5.3. Parallel data for grade 8 are displayed 
in Table 5.4. 

Typical instructional activities 

A snapshot of what students encounter on a weekly basis in the nation’s 
social studies classes can be developed by combining the data in the first two 
frequency categories — “Every day” and “Once or twice a week. Using this 
convention, it can be seen that, for grade 4, the greatest percentages of students 
encountered the following instructional activities on a weekly basis: using the 
social studies textbook (83 percent); using quantitative data, charts, or graphs 
(81 percent); completing worksheets (68 percent); hearing a teacher’s lecture 
(66 percent), and using books, newspapers or magazines (58 percent). Ten 
percent or less of students in grade 4 encountered the following instructional 
activities on a weekly basis: using computer software; writing a report of three 
or more pages; participating in debates or mock trials; or writing letters on 
civics topics. 

In Table 5.4, the pattern of frequency of use of instructional activities for 
grade 8 is much the same as it was for grade 4. The largest percentages of eighth 
graders also encountered the following instructional activities, at least on a 
weekly basis: using the social studies textbook (95 percent); using quantitative 
data, charts or graphs (77 percent); completing worksheets (76 percent); 
hearing a teacher’s lecture (77 percent); and, using books, newspapers or 
magazines (55 percent). 

In general, for the grade-8 students, the instructional activities encountered 
with low frequency on a weekly basis were similar to the pattern for grade-4 
students: using computer software; writing a report of three or more pages; 
participating in debates or mock trials; and writing letters on civics topics were 
all encountered by 12 percent or less of the students. 
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^ Percentoge is between 0.0 and 0.5. 

** ‘Sample is insufficient to permit a reliable estimate. 

NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1 998 Civics Assessment. 





Group work/ computer software, and film/video/filmstrips 

Matching students’ assessment results with the frequency of each instructional 
activity type allows for some estimate of the relationship between learning 
and various modes of instruction. Since many other variables (such as student 
ability grouping, teacher effectiveness, socioeconomic status, etc.) also affect 
instructional effectiveness, readers are cautioned not to attribute direct causality 
to the relationships described here. While most of the thirteen instructional 
activities shown in Tables 5.3 and 5.4 did not reach statistically significant 
relationships with student performance, a few did. For the most part, these 
statistically significant instructional activities were employed in the classroom 
setting with medium frequency (once to twice per week/month) compared with 
the other activities. 

It has been reported in a separate survey that social studies teachers seem less 
likely than teachers in other core academic subjects — English, mathematics, and 
science — to use alternatives to whole-class instruction.^ In other words, social 
studies teachers were less likely than mathematics teachers and science teachers 
to work with small groups. The data in tables 5.3 and 5.4 tend to support this 
finding. Only 35 percent of students in the fourth grade and 43 percent in the 
eighth grade received group activities or projects on at least a weekly basis. 
However, in both grades, small group activities were employed more commonly 
(43 percent in fourth grade and 52 percent in eighth grade) once- or twice- 
a-month — and the assessment results in grade 8 indicated that students in this 
frequency category had higher average civics scale scores than students whose 
teachers reported that they “Never or hardly ever” used small-group activity. 

In grade 4, both the occasional use of computer software and the use of 
quantitative data, charts, and graphs with any frequency in class were associated 
with higher scores on the civics assessment than for the groups that “never or 
hardly ever” used these instructional activities. In grade 8, the moderate (once 
to twice per week/month) use of films, videos, and filmstrips in class was 
associated with higher assessment scores. 



3 Ibid. 
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What are the Differences in Instructional 
Patterns Between Experienced and Less 
Experienced Teachers? 

The craft of effective teaching is developed over time as newer teachers try 
various instructional approaches with their various class levels. The NAEP 
teacher questionnaires address questions of the frequency of use of instructional 
activities, but they cannot capture other significant characteristics of teachers’ 
instructional techniques. While two teachers, for example, may use the same 
technology or materials with the same frequency to teach a given concept, one 
teacher may explain the concept differently or more effectively than the other. 
Further, as teachers grow in experience, they may be better able to choose the 
instructional activity that best fits a given educational goal. The National Board 
for Professional Teaching Standards (NBPTS) recommends that teachers be 
well grounded in the relative advantages and disadvantages of a large body of 
instructional activities and choose the most appropriate teaching strategies for 
a given lesson based on learning objectives, their students’ current level of 
knowledge, and available time and resources.'* 

Because the NAEP survey asked teachers to categorize their years of 
experience, their students’ performance can be examined by the interaction 
between teacher experience and type of instructional activity. Data were grouped 
for “less experienced” (2 years or less) and “experienced” (3 years or more) 
teachers. Table 5.5 displays the extent and pattern of significant differences 
found in favor of the experienced teachers for grade 4. Results are presented 
only where significant differences between less experienced and inexperienced 
teachers were observed. Each checkmark (i^) in the table indicates an analysis 
result in which either the percentage of students or the average scale score of 
students taught by experienced teachers was significantly greater than that of 
students taught by less-experienced teachers. The first row of the table (Social 
Studies Textbook), for example, shows that there were significant differences 
in three categories of teachers’ responses regarding frequency-of-use of a social 
studies textbook: the “Every day” category, the “Once or twice a week” 
category, and the “Once or twice a month” category. Within each category, the 
data and notation in the scale score column indicate that students of less 
experienced teachers were outperformed by students of more experienced 
teachers. Note that very few of the instructional activity categories differed in 
the frequency of instructional activity (percentage of students column). In fact, 
the experienced teachers tended to have greater percentages of students taught 
in the “once or twice a month” or the “never or hardly ever” category for the 



•* National Board for Professional Teaching Standards. (1989;. What teachers should know and be abU to do. 
Detroit, MI; Author 

National Board for Professional Teaching Standards. (1994). Early adcUsccnce/^eneralist standards for 
national board certification. Detroit, MI: Author 




CIVICS REPORT CARD • CHAPTERS 



112 



95 



Table 5.5 
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Summary of paired percentages of students and paired scale score 
means with significant differences between less experienced (two years 
or less) and experienced (three years or more) teachers by selected 
types of instructional activities, grade 4: 1998 



How ohea Jo yea me rite hdhwwg 
resoones to teadi sodal stwBes hi 
this dass? 

Social studios textbook 

Books, newspopers, magazines 

Primory documents 

Quantitative data, chorts, or grophs 

Computer software 

Rims, videos, filmstrips 

Hove students complete o worksheet 

Give a lecture 

Have students do o group activity/project 

Have students write o three or 
more page report 

Hove students portidpote in debates 
Have students portidpote in mock trials 
Have students write letters 



Frequency of instructional activity | 


Every doy 


Once or twice 
a week 


Once or twice 
amooth 


Never or 
hordly ever 



Percntoge torage P^w w iage Avmfe Pwiwrt a je Average Pcrcntogo 

of stodents sade score el students scab score of students scab score of students scho scare 











144-153 


141-151 


131-152 














148-155 


143-152 


138-157 












133-147 


144-154 


141-153 










142-153 


145-153 


144-155 
















12-25 


146-153 












8-16 


139-152 














143-151 


142-154 


130-154 










142-155 


140-152 




5-16 












140-154 


143-151 


138-153 
















150-156 














144-152 


57-70 141-153 














145-153 


140-155 














145-152 


150-156 



Indicates a significant difference in favor of experienced teachers. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1998 Civics Assessment. 
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instructional activities listed. In feet, for 9 out of 13 institutional activities, the 
students of experienced teachers who never used the activity outperformed the 
students of less experienced teachers who never used the activity. This finding is 
somewhat similar to the results of the NCES 1994-95 teacher follow-up survey 
which found that more experienced teachers were less likely than less experienced 
teachers to use some recommended teaching practices and more likely to use 
some traditional practices.^ 

In grade 4, the groups of students taught by teachers with three or more years 
of experience had higher civics scores than students taught by teachers with two 
years or less experience in every one of the instructional activities categories. As 
suggested earlier in this section, the difference between experienced and less 
experienced teachers in terms of student assessment performance appears to lie 
less in the choice of type or frequency of instructional activity than in what the 
teacher does within the activity. 

The relationships among teachers’ experience, ffequency/type of 
instructional activity, and student performances are complex and not easily 
predicted. Grade level also emerged as a factor in these relationships. Despite 
the large number of differences in performance for fourth-grade students by 
teacher experience and instructional type, very few comparable differences 
emerged in the data for grade 8 (data not displayed). As noted in the previous 
chapter, years of teacher experience was not significantly related to overall 
eighth-grade student performance on the civics assessment. 

Teacher professional development workshops 

Teachers may continue to learn about instructional activities that may be 
effective in their classrooms by attending professional development workshops 
or college courses. The teachers in grades 4 and 8 were asked the question, 

“7w the past twelve months, how many hours in total have you spent in professional 
development workshops or seminars in social studies or the teaching of social studies? 
Include attendance at professional meetings and conferences, district-sponsored or 
external workshops, and college or university courses.'" In grade 4, 58 percent of 
students were taught by teachers who had attended professional development 
activities in the past year. At grade 8, the parallel figure was 65 percent of the 
students. For purposes of this analysis, teacher responses were collapsed into 
those who checked “none” versus those who checked any of the other categories. 
This collapsed variable was then cross-tabulated with the instructional activities 
variables to shed some light on the pattern of possible differences in the classroom 
environment for teachers who attended professional development workshops or 
courses versus those who did not. 



5 Henke R. R. Chen X & Goldman, G. (1999). What happens in classrooms? Instructional practices in 
elementary a^ secondary schools, 1994-95. (NCES PubUcarion No. 1999-348). Washington, DC: Nanonal 
Center for Education Statistics. 
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Table 5.6 presents the results of these comparisons. Each check mark in the table 
represents a significant difference in the percentage of students receiving the indicated 
instructional activity based on whether their teachers had attended professional 
development. In reading Table 5.6, note that in the ‘‘used more” column, the higher 
percentage of students represents the workshop-attending teachers, while in the “used 
less” column the lower percentage of students represents the workshop-attending 
teachers. Some general patterns emerged fi-om the data. For example, eighth-grade 
teachers who attended workshops used textbooks less and taught greater 
percentages of students using more extended reports, debates, and mock trials. 
Fourth-grade teachers who attended workshops taught fewer students using 
worksheets and more students using group activities and the “active” instructional 
techniques of debates, mock trials, and letter writing, as well as external resources 
such as books, newspapers, and magazines. In addition, at both grades, teachers 
who attended professional development workshops taught more students using 
computer software than teachers who did not attend professional development 
workshops. The reader should keep in mind the results from the previous 
sections, which indicated that very low percentages of students were taught 
with the group-centered and student-active instructional activities. The results 
presented in this section indicate that I ) relatively low percentages of students 
encountered these instructional activities; and 2) teachers who attended 
professional development activities teach greater percentages of students using 
student-active teaching then teachers who had not attended professional 
development activities. 

Out-of-School Contexts 

Home support for learning is as important in social studies and civics as it is for other 
subjects surveyed in NAEP assessments. Variables related to the students’ home and 
community environments often show a strong relationship to performance. Such relation- 
ships were found in the 1998 NAEP assessments in reading and writing.*^ The following 
section examines some of these variables and their relation to NAEP civics scores. 

Discussion of schoolwork at home 

The extent to which schoolwork is discussed within the home is one indication of its 
priority for students and their families. When students discuss their schoolwork at 
home, they establish an important bond between school and home. Recent studies 
have documented the increased achievement of students whose parents are involved 
in their schooling.^ Civics topics lend themselves well to home discussion, since they 
are the basics for citizens regardless of their age. 

^ Donahue, P.L., Voclkl, K.E., Campbell, J.R. & Mazzco, J. (1999). NAEP 1998 reading report card for the nation 
and the states, (NCES Publication No. 1999-500). Washington, DC: National Center for Education Statistics. 

Greenwald, EA., Persky, H.R., Campbell, J.R., & Mazzco, J. (1999). NAEP 1998 writing report card for the 
nation and the states, (NCES Publication No., 1999-462). Washington, DC: National Center for Education 
Statistics. 

^ Christenson, S.L. (1992). Family feaors and student achievement: An avenue to increase students’ success. School 
Psychology Quarterly^ 7(3), 178-206. 
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Summary of examples of pairs of percentages of students with 
significant differences between social studies teachers who partici- 
pated in social studies workshops in the past year and those who 
did not, by type of instructiond activity, grades 4 and 8: 1998 






How oftai fbyoooseHio Mhmiog 
resoaxes to featb $odtd stofBes bi 



Grade 4 



Grade 8 



Used 

more 



Used 

less 









Sodal studies textbodc 








75-63 


Books, newspapers, magazines 


4-10 




34-52 




Primary documoits 


2340 




1-4 




Quantitative data, charts, or graphs 










Computer software 


13-30 




14-31 




Rims, videos, fifanstrips 










Have students complete a worksheet 




6-11 






Gve 0 lecture 










Have students do o group octivity/i^oject 


9-17 








Hove students write a three or 










more page report 






37-50 




Have students portidpate in debates 


20-32 




33-47 




Have students participate in mock trials 


24-34 




35-46 




Hove students write letters 


24-29 




16-31 





Indicates a significant difference in tfie percentage of students experiencing the instructional activity. 

NOTE: Teachers could rate the frequency of their use of the instructional activities above as: almost every 
doy, once or twice a week, once or twice a month, or never or hardly ever. The pairs of percentages in 
the table are representative of at least one difference within a frequericyof-use category. Often there was 
more than one significant difference per category. For simplicity of display, only one example (usually 
that with the largest percentages of students - not necessarily that with the largest difference) is shown. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1 998 Civics Assessment. 
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Students in the 1998 civics assessment were asked how often they discussed 
things they have studied in school (in any subject, not only civics) with someone 
at home. Their responses are summarized in Table 5.7. In every grade there 
is an association between frequency of discussion at home and average civics 
scale scores. The percentage of students reporting daily discussion appears to 
decline as grade level increases — from 53 percent at grade 4 to 33 percent 
at grade 12. However, the percentage of smdents who reported “never or 
hardly ever” discussing schoolwork at home remains fairly constant across 
grades (18 to 22 percent). About two-thirds or more of students across the 
three grades reported discussing schoolwork at home at least once a week 
(75 percent at grade 4, 67 percent at grade 8, and 64 percent at grade 12). 

In every grade, students who reported that they “never or hardly ever” 
discussed schoolwork at home had lower average scores than those who 
reported having some regular discussions. 



Table 5.7 
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Percentage of students, average civics scale scores, and percentage at 
or above Proficient by frequency of discussion of school studies at 
home, grades 4, 8, and 12: 1998 



How ohm do yoo Aans fbhtgs 
yo9 Imvo homed of sdnod odth 
someoae at hone? 


Percentage 
of students 


Average 
scale score 


Percentage dt or 
above Proficient 




Grade 4 




Doily 


53 


153 


26 


Once or twice a week 


22 


154 


26 


Once or twice a month 


7 


147 


19 


Never or hardly ever 


18 


140 


13 






Grade 8 


> 


Doily 


40 


157 


28 


Once or twice o week 


27 


153 


25 


Once or twice o month 


12 


147 


19 


Never or hardly ever 


21 


140 


13 






Grade 12 




Doily 


33 


157 


32 


Once or twice o week 


31 


154 


29 


Once or twice o month 


14 


150 


26 


Never or hardly ever 


22 


139 


16 



NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: National Center For Education Statistics, National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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Student volunteer work 

The past decade has seen increased emphasis on volunteering for community 
work — both among schoolchildren and adults. The welfare of the community 
is a topic inherent to civics education and, consequently, at grade 12 the 
students in the NAEP assessment were asked a question about whether they 
volunteered for work in their communities during the past year. Table 5.8 
presents the results. More than half (58 percent) indicated that they had done 
some volunteer work. The student volunteers were nearly evenly split between 
volunteering through their school and volunteering on their own (students 
could choose only one of the options). Twelfth-grade students who did 
volunteer work, whether through their school or on their own, had sigmficantly 
higher civics assessment scale scores than students who did not participate in 
volunteer work. 



Table 5.8 




Percentage of students, average civics scale scores, and percentage 
at or above Proficient by volunteer work status, grade 12: 1998 



THE HATKIH’S 
REPORT 



m: 



yoo do vokmteer wofk b 
yoar ioummUy tUi yeaf? 


■ 


Grade 1 2 


1 


Percentage 
of students 


Average 
scale score 


Percentage at or 
obove Proficient 


Yes, with my school 


27 


159 


35 


Yes, on my own 


31 


158 


34 


No 


43 


141 


16 



NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1998 Civics Assessment. 
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Students working at paid jobs 

Significant percentages of students in their final year of secondary school are 
working at jobs for pay. The Third International Mathematics and Science Study 
(TIMSS) found that twelfth-grade students in the United States reported 
working at a paid job for an average of 3.1 hours per “normal schoolday”* — 
for more than students in any of the other 20 countries in the study. Twelfth- 
grade students in the NAEP civics assessment were asked about the number 
of hoiu^ per week they worked at a job for pay, and the results are displayed 
in Table 5.9. Almost two-thirds (65 percent) of the twelfth-grade students 
indicated at least some hours worked each week. About a fifth (21 percent) 
of the students reported working more than 21 hours per week. Nearly half 
(49 percent) worked eleven or more hours per week. 



Table 5.9 
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Percentage of students, average civics scale scores, and percentage 
at or above Proficient by hours per week working at a job for pay, 
grade 12: 1998 



THE HglOrS 
REPORT 



Grade 12 



How many hoars do yoo Percentoge Average Percentage at or 

work at a fob hn pay? of students scale score above Proficient 



None 


36 


152 


30 


1-5 hours 


7 


155 


35 


6-10 hours 


9 


157 


34 


11-15 hours 


1 1 


159 


33 


1 6-20 hours 


17 


152 


25 


2 1 or more hours 


21 


142 


16 



NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 



^Mullis, I.V.S., Martin, M.O., Beaton, A.E., Gonzalez, E.J., Kelly, D.L. & Smith, Tj\, (1998). Mathematics 
and science achievement in the final year of secondary school: lEA^s third international mathematics and science 
study. Chestnut Hill, MA: Center for the Study of Testing, Evaluation, and Educational Policy, Boston 
College. 
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It is reasonable to assume that the more hours a student works, the fewer 
hours are available for homework and study. However, as was discovered in the 
TIMSS study, the relationship between hours worked and performance on the 
assessment is not simple and linear. It is not the case, for example, that students 
who did not work at all produced the highest scores. In the NAEP assessment, 
students who worked a moderate number of hours (between 6 and 15) per week 
had the highest scores on the assessment. Their scores were significantly above 
those for both the group who did not work and the groups who worked longer 
hours per week. Figure 5.1 displays this curvilinear relationship between hours 
worked and civics assessment average scores. The same pattern was found in the 
TIMSS study, where the subjects assessed were mathematics and science literacy. 



Figure 5.1 



Average civics scale scores by hours worked at a job for pay, 
grade 12: 1998 





Grade 12 



Hours worked 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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Summary 

This chapter focused on classroom contexts for learning social studies, 

interactions between teacher experience and instructional activities, and a 

selection of out-oTclassroom variables related to twelfth-grade students. 

Highlights of the findings include the following: 

► Over 70 percent of students at both grades 4 and 8 indicated that they had 
studied the U.S. Constitution and Congress in the current school year. 

► Generally less than half the fourth- and eighth-grade students indicated that 
they had studied other countries’ governments or international 
organizations such as the UN. 

► For both grades 4 and 8, the highest percentages of students were taught 
on a weekly basis with what might be termed “traditional” instructional 
activities: using the social studies textbook; using quantitative data, charts, 
or graphs; completing worksheets; hearing a teacher’s lecture; and using 
books, newspapers or magazines. 

► Instructional activities that were used on a weekly basis with low percentages 
of students included: use of computer software; writing a three or more page 
report; participating in debates or mock trials; and, writing letters on civic 
topics. 

► Only 35 percent of students in the fourth grade and 43 percent in the 
eighth grade received group activities or projects on at least a weekly basis. 
However, in both grades, small group activities were employed more 
commonly (53 percent in fourth grade and 52 percent in eighth grade) 

on a once- or twice-a-month basis — and the assessment results in grade 8 
indicated that students in this frequency category outscored students in the 
“Never or hardly ever” activity category. 
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^ The groups of students taught by experienced teachers in grade 4 had 
significandy higher average scores on the civics assessment in at least one 
ffequency-of-use-category for every one of the listed instructional activities. 
This pattern of advantage for experienced over less experienced teachers did 
not appear, however, in grade 8. 

^ In grade 4, 58 percent of students were taught by teachers who had attended 
professional development activities in the past year. At grade 8, the parallel 
figure was 65 percent. Teachers at grade 4 who attended workshops taught 
fewer students using worksheets and more students using group activities 
and the “active” instructional techniques of debates, mock trials and letter 
writing as well as external resources such as books and magazines. Teachers 
at grade 8 who attended workshops used textbooks less and taught greater 
percentages of students using more external source materials, extended 
reports, debates, and mock trials. 

^ In every grade there is a positive association between frequency of discussion 
of schoolwork at home and average civics scale scores. At least two-thirds of 
students across the three grades reported discussing schoolwork at home at 
least once a week. 

^ More than half of twelfth-grade students indicated they did some volunteer 
work in their communities. Those who volunteered had higher civics scores 
than those who never volunteered. 

^ Almost two-thirds of the twelfth-grade students indicated at least some 
hours worked each week at a job for pay. About a fifth (21 percent) of the 
students reported working more than 21 hours per week. Students who 
worked a moderate number of hours (between 6 and 15) had the highest 
scores on the assessment. 
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Overview of Procedures 
Used in the 

NAEP 1998 Civics Assessment 

Introduction 

This appendix provides information about the methods and procedures used 
in NAEP’s 1998 civics assessment. The NAEP 1998 Civics Technical Report 
contains more extensive information about these procedures. 

This NAEP report is based on results from six fourth-grade civics exercise 
blocks, eight eighth-grade civics exercise blocks, and eight twelfth-grade civics 
exercise blocks. In addition, there were two trend blocks at each grade. (Results 
of the trend study will appear in a separate report.) The assessment in each of 
the grades was conducted during the 1997-98 school year. More information 
about the composition of the civics assessment is presented below. 

Background of the Civics Assessment 

The 1998 civics assessment measured student achievement based on assessment 
objectives developed by nationally representative panels of civics educators and 
concerned citizens. The objectives for each assessment were based on the 
framework assessment developed by The Coimcil of Chief State School Officers 
in conjimction with the Center for Civic Education and the American Institutes 
for Research and approved by the National Assessment Governing Board to 
reflect content and process in school civics. That is, the objectives for the 1998 
civics assessment were not comparable to those of the 1988 assessment, and 
thus, results are not comparable to those of previous years. 
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The 1998 civics assessment contained multiple -choice and constructed- 
response questions measuring aspects of the framework and specifications. 

Each assessment booklet contained blocks of student background questions as 
well as civics cognitive questions. The civics assessment contained a range of 
questions measuring performance on sets of objectives developed by a nationally 
representative panel of citizens. The framework’s purpose was to provide a 
definition of civics on which to base the NAEP assessment. Developing this 
framework and the specifications that guided development of the assessment 
involved the critical input of many people, including representatives of national 
education organizations, teachers, parents, policymakers, business leaders, 
and members of the general public. This consensus process was managed by 
the Council of Chief State School Officers for the National Assessment 
Governing Board. 

NAEP previously assessed students’ performance in a civics assessment 
conducted during the school year ending in 1988, Because of the development 
of a completely new framework and specifications, direct comparisons between 
the results of the 1998 assessment and earlier assessment are not possible. 
However, two blocks of items from the 1988 assessment, based on the old 
framework, were readministered to a subsample of students at each grade in 
1998, making possible a 10-year comparison of performance on certain of the 
1988 items. The results will be published in a forthcoming separate report. 

The tasks required students to read and answer questions based on a variety 
of materials. The assessment was designed to evaluate students’ ability to recall 
specific information, make inferences based on an information passage or 
graphical stimulus (e.g., a political cartoon), or perform more analytical or 
evaluative tasks such as distinguishing opinion from fact or defending a position. 
The assessment administered at grade 4 included 90 items, 2 1 of them 
constructed-response, at grade 8, 151 items, 28 of them constructed response, 
and at grade 12, 152 questions, 29 of them requiring constructed responses. 
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The Design of the Civics Assessment 

The civics assessment consisted of eight different 25-minute segments or “blocks” 
of content questions at grades 8 and 12, and six at grade 4. Each also contained 
a small set of backgroimd questions that pertained to students’ experiences, 
instruction, and attitudes related to civics and to the testing experience. 

The cognitive blocks were assembled two to a booklet, together with a 
general background questionnaire, a civics background questionnaire, and a 
motivation block that were common to all booklets. The general background 
questionnaire included questions about demographic information and home 
environment. The blocks were placed in 32 booklets (18 at grade 4), each 
containing 15 items in grade 4, and all but one containing 19 items (the 
exception containing 18) in the other two grades. 

Sampling and Data Collection 

Sampling and data collection activities for the 1998 civics assessment were 
conducted by Westat, Inc. Based on procedures used since the inception of 
NAEP, the data collection for all three grades took place in the winter (January 
to March 1998). 

As with all NAEP national assessments, students in the civics assessment 
attending both public and nonpublic schools were selected for participation 
based on a stratified, three-stage sampling plan. The first stage included defining 
geographic primary sampling units (PSUs), which are typically groups of 
contiguous coimties, but sometimes a single county; classifying the PSUs 
into strata defined by region and community type; then selecting PSUs with 
probability proportional to size. In the second stage, both public and nonpublic 
schools are selected within each PSU that was selected at the first stage. The third 
stage involved randomly selecting students within a school for participation. 

(See the forthcoming NAEP 1998 Technical Report for further details.) A small 
number of students selected for participation was excluded because of limited 
English proficiency or severe disability. However, testing accommodations were 
offered to facilitate including as many of these students as possible. 
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The student sample sizes for the civics assessment, as well as the school and 
student participation rates, are presented in the following tables. The numbers 
in the tables are based on the full grade samples of students, taken when the 
grade samples were collected. Student sample sizes appear in Table A.l. School 
and student participation rates are shown in Table A. 2. Although sampled 
schools that refused to participate were replaced, school cooperation rates were 
computed based on the schools originally selected for participation in the civics 
assessment. The student participation rates represent the percentage of students 
assessed of those invited to be assessed, including those assessed in follow-up 
sessions when necessary. 



Table A.l 
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REPORT 
CARO 



NAEP civics sample sizes of main assessment and trend assessment, 
grades 4, 8, and 12; 1998 





Grade 4 


Grade 8 


Grade 12 


Moin assessment 


Assessed sample size 


5,948 


8,212 


7,763 


Excluded 


407 


341 


247 


'^'^1 rrenrf ossessmenf* j 


Assessed sample size 


2,088 


2,055 


2,193 


Excluded** 


176 


186 


103 



‘Results reported separately. 

“Accommodations were offered in the Main Civics Assessment, but were not available in the special trend study, 
so as to preserve comparability to 1 988. About half as many students in the main study were accommodated as 
were excluded. In the trend samples, most, but not all students who would have been offered accommodations had 
they been in the main study were excluded, leading to higher exclusion rates in the trend study. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 

1998 Civics Assessment. 
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NAEP civics school and student participation rates for the nation, 
grades 4, 8, and 12; 1998 








Grade 4 


Grade 8 


Grade 1 2 


Civics main assessment 


School participation rate 


88.6% 


84.6% 


78.0% 


Student participation rate 


94.8% 


92.3% 


79.4% 



SOURCE: NaHonal Center for Education Statistics, National Assessment of EducaHonal Progress (NAEP), 
1998 Civics Assessment. 
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The overall response rate (the product of the weighted school participation 
rate before substitution and the weighted student participation rate) for grade 
12 fell below the NCES reporting target of 70 percent. As a result, the 
background characteristics of both responding schools and all schools were 
compared to determine whether there was bias evident. The similarities in the 
distribution lend support to the conclusion that the data arc not seriously biased 
by these low response rates. 

Students with Disabilities (SD) and 
Limited English Proficient (LEP) Students 

It is NAEP’s intent to assess all selected students from the target population. 
Therefore, every effort is made to ensure that all selected students who are 
capable of participating in the assessment are assessed. Some students sampled 
for participation in NAEP can be excluded from the sample according to 
carefully defined criteria. These criteria were revised in 1996 to more clearly 
communicate a presumption of inclusion except under special circumstances. 
According to these criteria, students with Individualized Education Programs 
(lEPs) were to be included in the NAEP assessment except in the following cases: 

1. The school’s lEP team determined that the student could not participate, 
OR, 

2. The student’s cognitive functioning was so severely impaired that she or 
he could not participate. 

In cases where a student’s lEP required that the student be tested with an 
accommodation or adaptation and stated that the student could not demonstrate 
his or her knowledge without that accommodation, the student was provided with 
the appropriate accommodation. 

All LEP students receiving academic instruction in English for three years 
or more were to be included in the assessment. Those LEP students receiving 
instruction in English for less than three years were to be included unless school 
staff judged them incapable of participating in the assessment in English. 
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The reporting samples in the 1998 civics assessment used these criteria, with 
provisions made for accommodations. Students with disabilities or with limited 
English proficiency were included in the sample in the following way. At each 
grade one test booklet (two blocks) was designated to be the one administered 
to students requiring accommodations (the booklet was also administered to 
students not requiring accommodations). The booklet contained relatively few 
visual stimuli, for the sake of visually impaired students who might participate. 
Students were given accommodations that matched as closely as possible those 
provided them in other testing situations by their schools or instructors (most 
frequently, small group administration). Those students who did not typically 
need accommodations for testing were not provided with them. 

All the scale score and achievement level information in this report is 
based on a student sample that includes students who were provided with 
accommodations. The responses of students assessed with accommodations 
were evaluated according to the same criteria as those of students assessed 
without accommodations. Data on the individual questions presented in 
chapter 1, however, do not include responses from accommodated students 
because the questions only appeared in test booklets that were not administered 
to students requiring accommodations. 

Participation rates for the students with disabilities and LEP samples are 
presented in Table A.3 for all three grades. This table includes as the denominator 
the total number of all students who were identified for the assessment, including 
assessed and excluded students. The columns then show the raw numbers and 
weighted percentages of SD and LEP students who were identified for the 
assessment. The numbers and percentage are broken out by those excluded 
and those assessed, then further broken out into those assessed without 
accommodations and those assessed with accommodations. 



O 

ERIC 



112 



128 



1998 CIVICS REPORT CARD • APPENDIX A 



. . .. 



THE WOTON S 
REPORT 



m- 



Students with disabilities and limited English proficient students in 
NAEP civics assessment: National sample, public and nonpublic 
schools combined: 1998 





Grade 4 


Grade 8 


Grade 12 


Students tbot ore: 


Nwdirr of 
stideits 


WeisM 

perceotoge 
of students 
sodded 


Nondier ol 
students 


W^ted 
percentage 
ol students 
sonpled 


i 

Nomber ol ! 
students 


Weighted 
perceotoge 
of students 
sampled 




SD and LEP Students 


Identified 


1,064 


15 


1,099 


13 


759 


8 


Excluded 


407 


5 


341 1 

1 


4 


247 

1 


2 


Assessed 


657 


10 


758 


9 


512 


6 


Assessed without KCOiTunodatkms 


450 


6 


537 


6 


406 

1 


5 


Assessed with occonunodotions 


207 


3 


221 


3 


106 


1 




Students with disabilities 


Identified 


608 


11 


811 


11 


513 


6 


Excluded 


213 


3 


252 


3 


212 


2 


Assessed 


395 


7 


559 


8 


301 


* 4 


Assessed without QcoinmKidohons 


216 


4 


354 


- 4 


209 


3 


Assessed with ocomimodations 


179 


3 


205 


3 


92 


1 




Limited English proficient students 


Identified 


493 


5 


332 


3 


266 


2 


Excluded 


221 


2 


116 


1 


46 


▲ 


Assessed 


272 


3 


216 


2 


220 


2 


Assessed without octtunmodotim 


240 


2 


192 


2 


201 


2 


Assessed with ocomimodations 


32 


▲ 


24 


A 


19 


A 



^ Percentage is between 0.0 and 0.5. 

NOTE: The combined SD/LEP portion of the table is not a sum of the separate SD and LEP 
portions because some students were identified as both SD and LEP. Such students would be 
counted separately in the bottom portions but counted only once in the tap portion. Within 
each portion of the table, percentages may not sum properly due to rounding. 

SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1998 Civics Assessment. 
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Tabic A.4 displays the numbers and percentages of SD and LEP students 
assessed with the variety of available accommodations. It should be noted that 
students assessed with accommodations typically received some combination of 
accommodations. For example, students assessed in small groups (as compared 
to standard NAEP sessions of about 30 students) usually received extended time 
and had directions and/or assessment questions read aloud as needed. In one- 
on-one administrations, students often received assistance in recording answers, 
had directions and questions read aloud, and were afforded extra time. 
Extended time was considered the primary accommodation only when it 
was the only accommodation provided. 

Tables A. 4a and A.4b display the accommodations data for students with 
disabilities and LEP students, respectively. The denominator used to calculate 
the percentages for each of these tables is the total number of SD or LEP 
students assessed, as appropriate to the table. 



MM 






SD and LEP students assessed with and without accommodations, 
NAEP civics assessment: National sample, public and nonpublic 
schools combined: 1998 



THE NAHON’S 










, ' Grade 8 


Grad^ l2 ' 


Number of 
students 


Wdg[bted 
percentage of 
ossessed 
SDAEP 

students 


Number of 
students 


Wefghted 
percentage of 
ossess^ 

SDAEP 

students 


Number of 
students 


Weighted 
percentage of 
assessed 
SD/L£P 
students 


Told numb^ of 














assessed SD/UP stodents 


657 


100 


758 


100 


512 


100 


Assessed without occommodotioQs 


450 


64 


537 


65 


406 


78 


Assessed with occoimnodatloas 


207 


36 


221 


35 


106 


22 


Primary otcommodotion: 














Large print 


1 


A 


1 


A 


1 


A 


Extended time 


51 


8 


70 


11 


40 


8 


Read aloud 


6 


1 


9 


1 


1 


A 


Bilingud dictiomiry 


1 


A 


1 


A 


2 


A 


Small group 


125 


22 


128 


20 


54 


12 


One-otMine 


15 


3 


8 


1 


6 


1 


Scribe or computer 


3 


1 


2 


1 


0 


0 


Other 


5 


1 


2 


A 


2 1 


1 



^ Percentage is between 0.0 and 0.5. 

SD = Students with Disabilities (the term previously used was lEP) 

LEP = Limited English Proficient students 

NOTE: Percentages are based on total combined SD and LEP students assessed. 

The sum of percentages of students by primary accommodation may not total the overall 
percentage assessed with accommodations due to rounding. 

SOURCE, Natiorral Center for Education Statistics, National Assessment of 
EducaHonal Progress (NAEP), 1998 Civics Assessment, 
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Table A.4a 



Students with disabilities assessed with and without accommodations, 
NAEP civics assessment: National sample, public and nonpublic 
schools combined: 1998 



THE WAIIOII'S 
REPORT 



m: 





' Grade 4 






Grade 12- | 






Wdghted 




Weighted 




Weighted 






pertentoge of 




pmcentoge of 




percmitage of 






ossessed 




ossessed 




ossessed 




Number of 


SD 


Number of 


SD 


Number of 


SD 




students 


students 


students 


stmfonts 


students 


students 


Totd oonber of 










301 


100 


ossessed SD students 


395 


100 


559 


100 


Assessed without occommodotioos 


216 


56 


354 


60 


209 


71 


Assessed with occomnodotions 


179 


44 


205 


40 


92 


29 


Primary ououuBodatiotg 














LarQe pnnl 


1 


A 


1 


A 


1 


A 


Extended time 


39 


9 


60 


12 


34 


10 


Reoddoud 


4 


1 


8 


1 


1 


A 


Bihngud ifidionory 


1 


A 


0 


0 


0 


0 


Smdl group 


114 


29 


124 


24 


48 


16 


OneHUHme 


12 


3 


8 


2 


6 


2 


Scribe or OHiiput^ 


3 


1 


2 


1 


0 


0 


Other 


5 


1 


2 


A 


2 


1 



^ Percentage is between 0.0 and 0.5. 

SD = Students with Disabilities (the term previously used was lEP) 

NOTE: Percentages are based on total SD students assessed. 

The sum of percentages of students by primary accommodation may not total the overall 
percentage assessed with accommodations blouse of rounding. 

SOURCE: National Center for Education' Statistics, National Assessment of 
Educational Progress (NAEP), 1998 Civics Assessment. 
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Table A.4b 




Limited English proficient students assessed with and without 
accommodations, NAEP civics assessment: National sample, 
public and nonpubhc schools combined: 1998 



the MAnOW’S 
REPORT 



m. 





Grade 4 


Grade'S 


Grade 12 d| 








■ ^ 










Number of 


Weighted 
perxentoge of 
assessed 
LEP 


Number of 


Weighted 
percentage of 
ossessed 
LEP 


Number of 


Weighted 
perceoti^ of 
assessed 
LEP 




studmits 


students 


studmits 


students 


students 


students 


Ibfd nomber of 














assessed LEP students 


272 


100 


216 


100 


220 


100 


Assessed wHfM»t tKarnimodntioiis 


240 


89 


192 


90 


201 


92 


Assessed wHb occoamiodatioiis 


32 


11 


24 


10 


19 


8 


Primary accoamodatioK 














Large print 


0 


0 


0 


0 


0 


0 


Extency time 


12 


4 


12 


6 


8 


3 


Reodakxid 


2 


1 


3 


1 


0 


0 


B^nguoi didionflry 


0 


0 


1 


A ' 


2 


1 


Small group 


13 


5 


8 


3 


7 


3 


One-oiHme 


4 


2 


0 


0 


2 


1 


Sodte or congnitN 


0 


0 


0 


0 


0 


0 


Other 


1 


A 


0 


0 


0 


0 



^ Percentage is between 0.0 ond 0.5. 

LEP ° Limited English Proficient students 

NOTE: Percentoges ore bosed on totol LEP students ossessed. 

The sum of percentoges of students by primory occommodotion moy not totol the overoll 
percentage ossessed with occommodotions becouse of rounding. 

SOURCE: NoHonol Center for Educotion Statistics, Notionol Assessment of 
Educotionol Progress (NAEPj, 1998 Civics Assessment. 
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Scoring the Booklets 

Booklets from NAEP 1998 civics assessment were shipped to National 
Computer Systems (NCS) in Iowa City, Iowa, for processing. Receipt and 
quality control were managed through a sophisticated bar-coding and tracking 
system. After all appropriate materials were received from a school, they were 
forwarded to the professional scoring area, where the responses to constructed- 
response questions were evaluated by trained staff using guideUnes prepared by 
NAER Each constructed-response question had a unique scoring guide that 
defined the criteria to be used in evaluating students’ responses. Subsequent 
to the professional scoring, the booklets were scanned and all information 
was transcribed to the NAEP database at ETS. Each processing activity was 
conducted with rigorous quality control. An overview of the professional 
scoring follows. 

Scoring the Civics Constructed-Response Questions 

Most of the constructed-response questions were scored on a partial credit 
basis. The scoring guides identified the correct or acceptable answers for each 
question in each block. The scores for these questions included a 0 for no 
response and a 1 for an incorrect or “I don’t know” response. Completely 
correct answers received from 2 to 4 points, with intermediate scores awarded 
for varying degrees of partial credit. Because of the complex nature of the 
scoring, lengthy training was required. In an orientation period, the readers 
were trained to follow the procedures for scoring the questions and given an 
opportunity to become familiar with the scoring guides. 

The 1998 grade 8 assessment included 28 questions for which students were 
required to construct written responses. The scoring guides for the constructed- 
response questions focused on students’ abUity to perform various tasks: for 
example, identifying the intended message of a poster and substantiating their 
interpretations with examples. The scoring giudes for the questions varied 
somewhat, but typically included the distribution of score points shown below 

in Table A.5. 
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Labels for score levels of polytomous items, NAEP civics assessment: 



1998 



Score 


3-category item 


4-category item 


4 




Complete 


3 


Complete 


Acceptable 


2 


Partial 


Partial 


1 


Unacceptable 


Unacceptable 


0 


Off-task, or omitted 


Off-task, or omitted 



NOTE: The categories fatting between "Unacceptable" and "Complete" represent increasing levels of a 
partially correct response. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1998 Civics Assessment. 



The training program for the assessment scoring was carried out on all 
assessment questions, one at a time, for each age group and covered the range 
of student responses. The actual training was conducted by ETS staff assisted 
by NCS’s scoring director and team leaders. Training began with each reader 
receiving a photocopied packet of materials consisting of a scoring guide, a set 
of 10-15 anchor papers, and an additional 15-20 response samples to be scored 
by the reader for practice. The trainers reviewed the scoring guide with the 
readers, explained all the applicable score points, and elaborated on the rationale 
used to arrive at a particular score. The readers then reviewed the anchor 
papers, as the trainers clarified and elaborated on the scoring guide. After this 
explanation, the practice samples were scored and discussed until the readers 
were in agreement. If necessary, additional packets of 1998 responses were used 
for practice scoring. 
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After some further discussion, scoring of the 1998 responses began. If 
scoring of a question ran over to a second day, the readers reviewed the scoring 
guides and scored 10-20 “qualification” sample papers before resuming scoring. 

Real-time reliability studies were conducted as part of this scoring, and the 
results fed back to scoring table leaders to monitor and improve results through 
rescoring and retraining. For the 1998 material, 25 percent of the constructed 
responses were scored by a second reader to produce interreader reliability 
statistics. The reliability information from these studies is shown in Table A.6. 

wnoN’s 
ivaep 

fe- 

Interrater reliabilities for scoring of civics open-ended items, grades 
4, 8, and 12: 1998 






Grade 4 


Grade 8 


Grade 12 


Percent exact agreement 


89 


87 


85 


Range of percent agreement 


81-98 


68-96 


72-93 


Reliability coefficient 


.897 


.895 


.896 



SOURCE; Notionol Center for Educotion Statistics, Notionol Assessment of Educotionol Progress (NAEP), 

1 998 Civics Assessment. 

Data Analysis and IRT Scaling 

After the assessment information had been compiled in the NAEP database, 
the data were weighted according to the sample design and the population 
structure. The weighting for the samples reflected the probability of selection 
for each student as a result of the sampling design, adjusted for nonresponse. 
Through poststratification, the weighting assured that the representation of 
certain subpopulations corresponded to figures from the U.S. Census and the 
Current Population Survey. 

Analyses were then conducted to determine the percentage of students 
who gave various responses to each cognitive and background question. Item 
response theory (IRT) was used to estimate average proficiency for the nation 
and for various subgroups of interest within the nation. IRT scaling was 
performed separately within each grade level for each of the three civics 
assessment grades. 

IRT models the probability of answering a question correctly as a 
mathematical function of proficiency or skill. The main purpose of IRT analysis 
is to provide a common scale on which performance can be compared across 
groups, such as those defined by age, assessment year, or subpopulations 
(e.g., race/ethnicity or gender). 
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Students do not receive enough questions about a specific topic to permit 
reliable estimates of individual performance. Traditional test scores for 
individual students, even those based on IRT, would contribute to misleading 
estimates of population characteristics, such as subgroup averages and 
percentages of students at or above a certain proficiency level. Instead, NAEP 
constructs sets of plausible values designed to represent the distribution of 
proficiency in the population.^ A plausible value for an individual is not a scale 
score for that individual but may be regarded as a representative value from 
the distribution of potential scale scores for all students in the population 
with similar characteristics and identical patterns of item response. Statistics 
describing performance on the NAEP scales are based on these plausible values. 
These statistics estimate values that would have been obtained had individual 
proficiencies been observed, that is, had each student responded to a sufficient 
number of cognitive questions so that his or her proficiency could be precisely 
estimated. For the 1998 civics exercises, a single IRT scale was constructed for 
each grade. These scales had identical means and standard deviations, so that 
reflecting the assessment design with no common items across grades, cross- 
grade comparisons are neither meaningful nor possible. 

For the civics assessment, a scale ranging from 0 to 300 was created, using 
a generalized partial-credit (GPC) model. ^ Developed by ETS and first used 
in 1992, the GPC model permits the scaling of tasks scored according to 
multi-point rating schemes. The model takes full advantage of the information 
available from each of the student response categories used for these more 
complex performance tasks. 

As described earlier, the NAEP scales for all the subjects make it possible 
to examine relationships between students’ performance and a variety of 
background factors measured by NAEP. The fact that a relationship exists 
between achievement and another variable, however, does not reveal the 
underlying cause of the relationship, which may be influenced by a number of 
other variables. Similarly, the civics assessment does not capture the influence 
of unmeasured variables. The results are most useful when they are considered 
in combination with other information about the student population and the 
educational system, such as changes in instruction, changes in the school-age 
population, and societal demands and expectations. 



^ For theoretical justification of the procedures employed, see Mislevy, R.J. (1988). Randomization- based 
inferences about latent variables fi*om complex samples. Psychonutrtka^ 56 (2), 177-96. 

For computational details, see the forthcoming NAEP 1998 Technical Report. 

2 Muraki, E. (1992). A generalized partial-credit model: Application of an EM algorithm. Applied 
Psychological Measurement., 16{2). 159-176. 
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NAEP Reporting Groups 

This report contains results for the nation and for groups of students within 
the nation defined by shared characteristics. The subgroups defined by race/ 
ethnicity, parents’ education level, gender, and region are defined below. 

Race/Ethnicity 

Results are presented for students in different racial/ethnic groups according 
to the following mutually exclusive categories: White, Black, Hispanic, Asian/ 
Pacific Islander, and American Indian (including Alaskan Native). Some racial/ 
ethnic results are not reported separately because there were too few students 
in the group. The data for all students, regardless of whether their racial/ 
ethnic group was reported separately, were included in computing the overall 
national results. 

Two questions from the set of general student background questions were used 
to determine race/ethnicity: 

If you are Hispanic, what is your Hispanic background.> 

□ I am not Hispanic. 

□ Mexican, Mexican American, or Chicano 

□ Puerto Rican 

□ Cuban 

□ Other Spanish or Hispanic background 

Students who responded to this question by filling in the second, third, 
fourth, or fifth oval were considered Hispanic. For students who filled in the 
first oval, did not respond to the question, or provided information that was 
illegible or could not be classified, responses to the following question were 
examined to determine their race/ ethnicity: 

Which best describes you.> 

□ White (not Hispanic) 

□ Black (not Hispanic) 

□ Hispanic (“Hispanic” means someone who is Mexican, Mexican 
American, Chicano, Puerto Rican, Cuban, or from some other Spanish 
or Hispanic background.) 

□ Asian or Pacific Islander (“Asian or Pacific Islander” means someone 
who is from a Chinese, Japanese, Korean, Filipino, Vietnamese, Asian 
American, or from some other Asian or Pacific Island background.) 

Q American Indian or Alaskan Native (“American Indian or Alaskan 

Native” means someone who is from one of the American Indian tribes, 
or one of the original people of Alaska.) 

□ Other (specify) 
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Parents^ Education Level 

Students were asked to indicate the extent of schooling for each of their parents: 
did not finish high school, graduated from high school, had some education 
after high school, or graduated from college. The response indicating the higher 
level of education for either parent was selected for reporting. 

Students were asked to indicate the extent of their mother’s education by 
answering the following three questions: 

Did your mother graduate from high school? (“Mother” can be a mother, 
stepmother, or female guardian.) 



□ Yes 

□ No 

□ I don’t know. 



Did your mother have some education after high school? (“Mother” can be a 
mother, stepmother, or female guardian.) 



□ Yes 

□ No 

□ I don’t know. 



Did your mother graduate from college? (“Mother” can be a mother, 
stepmother, or female guardian.) 

□ Yes 

□ No 

□ I don’t know. 

Students were asked the same three questions about their father’s education 
level, as shown below: 

Did your father graduate from high school? (“Father” can be a father, 
stepfather, or male guardian.) 

□ Yes 

□ No 

□ I don’t know. 



Did your father have some education after high school? (“Father” can be a 
father, stepfather, or male guardian.) 

□ Yes 

□ No 

□ I don’t know. 
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Did your father graduate from college? (“Father” can be a father, stepfather, or 
male guardian.) 

□ Yes 

□ No 

□ I don’t know. 



Gender 

Results are reported separately for males and females. Gender was reported by 
the student. 



Region 

The United States was divided into four regions: Northeast, Southeast, Central, 
and West. States in each region are shown on the map below. Each state except 
Virginia is contained entirely in one region. The part of Virginia that is part 
of the Washington, D.C. -Maryland- Virginia metropolitan statistical area is 
included in the Northeast region; the remainder of the state is included in the 
Southeast region. 



Figure A.l 



■ 



States included in the four NAEP regions 



THE MAnON’S 
REPORT 



m- 



Northeast 



Connecticut 

Delaware 

District of Columbia 
Maine 
Maryland 
Massachusetts 
New Hampshire 
New Jersey 
New York 
Pennsylvania 
Rhode Island 
Vermont 
’^Virginia 



Southeast 



Alabama 
Arkansas 
Florida 
Georgia 
Kentucky 
Louisiana 
Mississippi 
North Carolina 
South Carolina 
Tennessee 
’^Virginia 
West Virginia 



Central 



Illinois 

Indiana 

Iowa 

Kansas 

Michigan 

Minnesota 

Missouri 

Nebraska 

North Dakota 

Ohio 

South Dakota 
Wisconsin 



Alaska 

Arizona 

California 

Colorado 

Hawaii 

Idaho 

Montana 

Nevada 

New Mexico 

Oklahoma 

Oregon 

Texas 

Utah 

Washington 

Wyoming 



The port of Virginia that Is included in the Washington, DC metropolitan area is included in the Northeast region; 
the remainder of the state is included in the Southeast region. 

SOURCE: Nartonal Center for Education Statistics, National Assessment of Educational Progress (NAtP), 

1 998 Civics Assessment. 




1 998 CMCS REPORT CARD • APPENDIX A 



139 



123 



Weighting and Variance Estimation 

A complex sample design was used to select the students who were assessed. The 
properties of a sample selected through a complex design could be very different 
from those of a simple random sample, in which every student in the target 
population has an equal chance of selection and in which the observations from 
different sampled students can be considered to be statistically independent of one 
another. Therefore, the properties of the sample for the complex data collection 
design were taken into account during the analysis of the assessment data. 
Standard errors calculated as though the data had arisen from a simple random 
sample would generally underestimate sampling error. 

One way that the properties of the sample design were addressed was by using 
sampling weights to account for the fact that the probabilities of selection were 
not identical for all students. All population and subpopulation characteristics 
based on the assessment data used sampling weights in their estimation. These 
weights included adjustments for school and student nonresponse. 

The statistics presented in this report are estimates of group and subgroup 
performance based on samples of students, rather than the values that could be 
calculated if every student in the nation answered every assessment question. 

It is therefore important to have measures of the degree of uncertainty of the 
estimates. Accordingly, in addition to providing estimates of percentages of 
students and their average scale score, this report provides mformation about 
the uncertainty of each statistic. 

Two components of uncertainty are accounted for in the variability of 
statistics based on scale scores: the uncertainty due to sampling only a small 
number of students relative to the whole population and the uncertainty due 
to sampling only a relatively small number of questions. The variability of 
estimates of percentages of students having certain background characteristics 
or answering a certain cognitive question correctly is accounted for by the 
first component alone. Because NAEP uses complex sampling procedures, 
conventional formulas for estimating sampling variability that assume simple 
random sampling are inappropriate. For this reason, NAEP uses a jackknife 
replication procedure to estimate standard errors due to sampling from means 
of paired primary sampling units (PSU’s). The jackknife standard error provides 
a reasonable measure of uncertainty for any information about students that 
can be observed without error, but each student typically responds to so few 
questions within any content area that the scale score for any single student 
would be imprecise. In this case, using plausible values technology makes it 
possible to describe the performance of groups and subgroups of students, but 
the underlying imprecision that makes this step necessary adds an additional 
component of measurement variability to statistics based on NAEP scale scores. 
Normally, each analysis is done once for each available plausible value, and the 
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variance of the (usually 5) resulting statistics estimated with the jackknife 
technique. To replicate the analyses reported here, the secondary analyst must 
have access to both the plausible values and replicate weights employed. 

Licenses to use these data are available by application to NCES. 

The reader is reminded that, like those from all surveys, NAEP results 
are also subject to other kinds of errors, including the effects of necessarily 
imperfect adjustments for student and school nonresponse and other largely 
unknowable effects associated with the particular instrumentation and data 
collection methods used. Nonsampling errors can be attributed to a number 
of sources: inability to obtain complete information about all selected students 
in all selected schools in the sample (some students or schools refused to 
participate, or students participated but answered only certain questions); 
ambiguous definitions; differences in interpreting questions; inability or 
unwillingness to give correct information; mistakes in recording, coding, or 
scoring data; and other errors of collecting, processing, sampling, and estimating 
missing data. The extent of nonsampling errors is difficult to estimate. By their 
nature, the impacts of such error cannot be reflected in the data-based estimates 
of uncertainty provided in NAEP reports. 

Drawing Inferences from the Results 

Confidence Intervals Around Means and Percentages 

The use of confidence intervals, based on the standard errors, provides a way 
to make inferences about the population averages and percentages in a manner 
that reflects the uncertainty associated with the sample estimates. An estimated 
sample scale score average ± 2 standard errors represents about a 95 percent 
confidence interval for the corresponding population quantity. This means that 
with 95 percent certainty, the average performance of the entire population of 
interest is within about ± 2 standard errors of the sample average. 

As an example, suppose that the average scale score of students in a 
particular group was 156, with a standard error of 1.2. A 95 percent confidence 
interval for the population quantity would be as follows: 

Average ± 2 standard errors = 156 ± 2 (1.2) = 156 ± 2.4 = 

156 - 2.4 and 156 + 2.4 = (153.6, 158.4) 

Thus, one can conclude with close to 95 percent certainty ±at the average 
scale score for the entire population of students in that group is between 153.6 
and 158.4. 

Similar confidence intervals can be constructed for percentages, provided 
that the percentages are not extremely large or extremely small. For percentages, 
confidence intervals constructed in the manner above work best when sample 
sizes are large, and the percentages being tested have magnitude relatively close 
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to 50 percent. Statements about group differences should be interpreted with 
caution if at least one of the groups being compared is small in size and/ or if 
“extreme” percentages are being compared. Percentages, P, were treated as 
“extreme” if: 

p<p = — 200 , where the effective sample size is N^.. = P(100 ~ P\ ^nd S£,> 

N +2 (SEf 

is the jackknife standard error of P. 

This “rule of thumb” cutoff leads to flagging a large proportion of 
confidence intervals that would otherwise include values less than 0 or greater 
than 100. In either extreme case, the confidence intervals described above are 
not appropriate, and procedures for obtaining accurate confidence intervals are 
quite complicated. In this case, the value of Pwas reported, but no standard 
error was estimated and hence no tests were conducted. 

As for percentages, confidence intervals for average scale scores are most 
accurate when sample sizes are large. For some of the subgroups of students for 
which average scale scores or percentages were reported, student samples sizes 
could be quite small. For results to be reported for any subgroup, a minimum 
student sample size of 62 was required. If students in a particular subgroup were 
clustered within a small number of geographic primary sampling units (PSUs), 
the estimates of the standard errors might also be inaccurate. So, subgroup data 
were required to come from a minimum of five PSUs. 

Comparing Means and Percentages of Different Groups 

Flow large does a numeric difference in NAEP data have to be in order not to 
be a merely chance occurrence? Drawing appropriate and justifiable inferences 
fi-om the data gathered in the NAEP assessment is a particularly complex 
statistical endeavor because: I) a sampling design is used for data collection, 
and 2) no single student takes more than a section of the entire question pool. 
Judging the statistical differences between subgroups in the NAEP data requires 
an understanding of the following: confidence intervals (as noted above) and 
standard errors in the sampling design, the application of the t-test paradigm, 
the notion of degrees of fi^eedom as applied in a stratified sample, and the 
application of the concepts of statistical family size in the context of making 
multiple comparisons. Each of these topics will be touched upon in the 
following sections. Because of these complexities, quick “rule of thumb” 
estimates of the significance of differences among, for example, scaled scores 
in NAEP subgroups in the data tables in this report cannot be relied upon to 
be accurate. 
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T-testS. In some cases, the differences between groups were not discussed 
in this report. This happened for one of two reasons: (a) if the comparison 
involved an extreme percentage (as defined above); or (b) if the standard error 
for either group was subject to a large degree of uncertainty (i.e., the coefficient 
of variation is greater than 20 percent, denoted by “***” in the tables).^ In 
either case, the results of any statistical test involving that group needs to be 
interpreted with caution, and so the results of such tests are not discussed in 
this report. 

Among the major findings reported for NAEP assessments are mean 
differences between groups, for example, comparisons of public and private 
school students. Such comparisons are assessed for statistical significance by 
a t-test of the form: 



Where: 

m. and m. are the means for groups i and j, and and are the jackknife 
estimates of sampling variance for groups i and j. The reader should note 
that this procedure uses a conservative estimate of the standard error of the 
difference (i.e., one that may overstate sampling variability), since the estimates 
of the group averages or percentages will be positively correlated to an unknown 
extent due to the sampling plan. However, since sources of survey error other 
than sampling (e.g., error associated with item parameter estimation and the 
error associated with linking results across years) are not accounted for in 
the standard errors, using these conservative estimates has been considered 
advisable. Moreover, direct estimation of the standard errors of all reported 
differences would involve too heavy a computational burden to be implemented 
in practice. 

Effective degrees of freedom. Because of the clustered nature of 
the sample, the “effective degrees of freedom” for this t-test is considerably 
less than the number of students entering into the comparison, and, indeed, 
less than the number of PSU pairs that go into its computation. (See the 
forthcoming NAEP 1998 Technical Report for more details.) The degrees 



3 As was discussed in the section “Weighting and Variance Estimation,” estimates of standard errors subject 
to a large degree of uncertainty are designated by the symbol “***”. In such cases, the standard error 
and any confidence intervals or significance tests among these standard errors should be interpreted 
with caution. 
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of freedom of this t-test are estimated by an approximation given by 
Johnson & Rust^ as follows: 




where the summation is over the two groups being compared. The item, is 
the degree of freedom estimate for the variance of the mean and is defined 
by Satterthwaite® with a correction term suggested by Johnson and Rust. It is 
derived by matching estimates of the first two moments of the variance to those 
of a chi-square random variable. 



Here, j stands for jackknife replicate j, and the summations are over all 
replicates, usually 62 in NAER The term is the mean of subgroup k for 
the jth jackknife replicate. The term is the overall mean for subgroup k 
using the overall weights and the first plausible value. 

The number of degrees of freedom for the variance equals the number of 
independent pieces of information used to generate the variance. In the case 
of data from NAEP, the pieces of information are the 62 squared differences 
{m.^- each supplying at most one degree of freedom (regardless of how 
many individuals were sampled within PSUs). If some of the squared differences 
{m.^- are much larger than others, the variance estimate of is 
predominantly estimating the sum of these larger components, which dominate 
the remaining terms. The effective degrees of freedom of in this case will 
be nearer to the number of dominant terms. The estimate, df^,, reflects 
these relationships. 

The two formulae above show us that when df^ is small, the degrees of 
freedom for the t-test, df, wiU also be smaU. This wiU tend to be the case when 
only a few PSU pairs have information about subgroup differences relevant to 
a t-test. It will also be the case when a few PSU pairs have subgroup differences 
much larger than other PSU pairs. With a clustered sample and a practical 
sample size, it is not possible to stratify over every group of potential interest. 

It is thus inevitable that in a particular assessment for some groups, some pairs 



* Johnson, E. & Rust, K. (1992). “Effective Degrees of Freedom for Variance Estimates from a Complex 
Sample Survey,” Proceedings of the Section on Survey Research Methods, Amencan Statistical Associanon, 
863-866. 

5 Satterthwaite, F.E. (1941). “Synthesis of Variance,” Psychometrika 16, 5, 309-316. 
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within some PSU’s will be less well-matched than others. Because this depresses 
the estimate of effective degrees of freedom, it has a conservative effect on 
declaring significance for comparisons involving those groups (see p. 35). 

Conducting multiple tests. In rnany applications of significance testing, 
the t-test value is compared to a t-distribution with effective degrees of freedom, 
as given in the previous section, and is assessed at a nominal level of .05/2 for a 
two-tailed test. However, in most sections of this report, many different groups 
are compared (i.e., multiple sets of means are being analyzed). In sets of 
comparisons, statistical theory indicates that the certainty associated with 
the entire set of comparisons is less than that attributable to each individual 
comparison from the set. To hold the sigmficance level for the set of 
comparisons at a particular level (e.g., .05), adjustments (called “multiple 
comparison procedures”*) must be made to the methods described in the 
previous section. One such procedure, the False Discovery Rate (FDR) 
procedure,^ was used to control the certainty level. 

Unlike the other multiple comparison procedures (e.g., the Bonferroni 
procedure) that control the familywise error rate (i.e., the probability of making 
even one false rejection in the set of comparisons), the FDR procedure controls 
the expected proportion of falsely rejected hypotheses. Rather than holding the 
probability, a, of even one true null hypothesis being rejected at some level, 
usually .05, as does the Bonferroni procedure by setting the critical value for 
significance at .05/m, where m is the number of comparisons in the set, or 
“family size,” the Benjamini and Hochberg False Discovery Rate controls the 
expected proportion of true null hypotheses declared significant. The FDR = 
Expectation (V/R), where V = Number of true null hypotheses declared 
significant, and R = Total number of hypotheses declared significant. This is 
accomplished by ordering the contrasts from most probable to least probable, 
and testing sequentially, with a systematically decreasing value of a until a null 
hypothesis is rejected. That hypothesis and all subsequent hypotheses, which 
have smaUer probability, are declared significant. FamUywise procedures are 
considered conservative for large families of comparisons.* Therefore, the FDR 
procedure is more suitable for multiple comparisons in NAEP than other 
procedures. A detailed description of the FDR procedure appears in the 
forthcoming NAEP 1998 Technical Report. 



6 Miller, R.G. (1966). Simultaneous statistical inference. New York: Wiley. 

7 Benjamini Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical ^werful 
approach to multiple testing. Journal of the Royal Statistical Society, Series B, No. i, pp. 298-300. 

8 Williams V.S.L., Jones, L.V., & Tukey, J.W. (1994, December). ControUins error in multipU comparisons 
with special atUntion to the National Assessment of Educational Prosress. Research Triangle Park, NC: 
National Institute of Statistical Sciences. 
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The 1998 assessment is the first time NAEP has used the Benjamini- 
Hochberg procedure to maintain FDR for all multiple comparisons. Prior to the 
1996 assessment, the Bonferroni procedure was used for multiple comparisons. 
Beginning in the 1996 assessment, the Bonferroni procedure was used for 
multiple comparisons involving small sets of intervals, and FDR for large sets. 

The Benjamin and Flochberg FDR application consists of arranging the 
m significance tests in order, from lowest to highest probability P(l) < P(2) ... 

< P(w). To obtain an overall control of the False Discovery Rate at .05, 
compare P(w) — the largest probability with .05. If P(w) < .05, declare all m 

comparisons significant. If not, compare P(w- 1) with ^ ^ . ( 05) . If less 

YU 

than or equal to this criterion, declare P(l) - P( w - 1) significant. If not, follow 
the same procedure with P(w - 2) down through P(l). Note that P(l) ^ 

— . (.05) is the level used in the Bonferroni procedure for all comparisons. 
m 

When we conduct multiple comparisons, the family size, or number of 
comparisons in a set, is a crucial issue. The investigator has a choice between 
prespecifying a limited number of comparisons, and thus maintaining power, or 
looking at many contrasts sacrificing power for the chance of finding additional 
significant contrasts. For example, the Parents’ Flighest Level of Education 
Variable has five categories: 

Graduated from college 
Some education after high school 
Graduated from high school 
Did not finish high school 
I don’t know. 

If we choose Graduation from college as the target category for the set, and 
contrast mean proficiency of each of the other four categories with the target, 
our family size is four, and under FDR, any mean difference with probability 

< .05/4 = .0125 will certainly be declared significant. If, on the other hand we 
want to test all possible contrasts, family size becomes ten, and significance in 
the lowest probability category is guaranteed only if its probability < .005. This 
tension between conserving power and identifying a larger number of significant 
comparisons at the risk of not finding any requires careful thought and 
balancing. After much discussion, it has been determined that NAEP should 
use all possible comparisons within a set in conducting multiple comparisons. 
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Cautions in Interpretations 

As described earlier, the NAEP civics scale makes it possible to examine 
relationships between students’ performance and various background factors 
measured by NAEP. However, a relationship that exists between achievement 
and another variable does not reveal the underlying cause of the relationship, 
which may be influenced by a number of other variables. Similarly, the 
assessments do not capture the influence of unmeasured variables. The results 
are most useful when they are considered in combination with other knowledge 
about the student population and the educational system, such as trends 
in instruction, changes in the school-age population, and societal demands 
and expectations. 

Grade- 12 Participation Rates and Motivation 

NAEP has been described as a “low-stakes” assessment. That is, students receive 
no individual scores, and their NAEP performance has no effect on their grades, 
promotions, or graduation. There has been continued concern that this lack of 
consequences affects participation rates of students and schools, as well as the 
motivation of students to perform well on NAEP. Of particular concern has 
been the performance of twelfth graders, who typically have lower student 
participation rates than fourth and eighth graders and who are more likely to 
omit responses compared to the younger cohorts. 

Participation Rates 

In NAEP, there has been a consistent pattern of lower participation rates for 
older students. The participation rate in civics for grade 12 students was 
79 percent, compared with 95 percent for grade 4 students and 92 percent for 
grade 8 students. School participation rates (the percentage of sampled schools 
that participated in the assessment) have also typically decreased with increasing 
grade level. The school participation rate was 89 percent for the fourth grade, 
85 percent for the eighth grade, and 82 percent for the twelfth grade. 

The effect of participation rates on student performance, however, is 
unclear. Students may choose not to participate in NAEP for many reasons, 
such as a desire to attend regular classes so as not to miss important instruction 
or fear of not doing well on NAEP. Similarly, there are a variety of reasons 
for which various schools do not participate. The sampling weights and 
nonresponse adjustments, described earlier in this appendix, provide an 
approximate statistical adjustment for nonparticipation. However, the effect 
of some school and student nonparticipation may have an undetermined effect 
on results. 
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Motivation 



To the extent that students in the NAEP sample are not trying their hardest, 
NAEP results may underestimate student performance. The concern increases 
as students get older and is particularly pronounced for twelfth graders. The 
students themselves furnish some evidence about their motivation. As part of 
the background questions, students were asked how important it was to do well 
on the NAEP writing assessment. They were asked to indicate whether it was 
very important, important, somewhat important, or not very important to 
them. The percentage of students indicating they thought it was either 
important or very important to do well was 88 percent for fourth graders, 

59 percent for eighth graders, and 33 percent for twelfth graders. 

Several factors may contribute to this pattern. NAEP was administered in 
the late winter, when high school seniors often have other things on their 
minds. Another factor that may have contributed to lack of motivation is the 
fact that the civics assessment consists pardy of constructed-response questions 
which tend to be more time-consuming than multiple -choice questions. As 
with participation rates, however, the combined effect of these and other factors 
is unknown. 

It is also interesting to note that students who indicated it was very 
important for them to do well on NAEP did not have the highest average 
scores. These data further cloud the relationship between motivation and 
performance on NAEP. 

Need for Future Research 

More research is needed to delineate the factors that contribute to nonparticipation 
and lack of motivation. To that end, NCES plans to commission a study of high 
school transcripts to learn more about the academic performance of twelfth-grade 
students who do not participate in the assessment. In addition, NCES is currently 
investigating how various types of incentives can be effectively used to increase 
participation in NAEP. 
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The comparisons presented in this report are based on statistical tests that 
consider the magnitude of the difference between group averages or percentages 
and the standard errors of those statistics. This appendix contains the standard 
errors for the estimated averages and percentages in all the tables and figures 
throughout this report. Because NAEP scores and percentages are based on 
samples rather than the entire population(s), the results are subject to a measure 
of uncertainty reflected in the standard errors of the estimates. It can be said 
with 95 percent certainty that for each population of interest, the value for the 
whole population is within plus or minus two standard errors of the estimate for 
the sample. 




Standard errors for civics scale scores by percentiles; 1998 
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SOURCE: National Center for Education Statistics, National Assessment of Educational 
Progress (NAEPj, 1 998 Civics Assessment. 
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Standard errors for percentage of students at or above the civics 
achievement levels for the nation: 1998 



Nation 



Below 

Basic 


At or 
above 
Basic 


At or above 
Proficient 


Advanced 


“ y.' 


Gracie 4 


W 1 


1.0 




0.9 


0.3 




j Grade 8. 




0.9 


0.9 


0.8 


0.2 




1 Grade 1 2 




0.9 


0.9 


0.9 


0.4 



SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 




Standard errors for percentage of students within each civics 
achievement level range for the nation: 1998 
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SOURCE: National Center for Education Statistics, 
Natior>ai Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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Standard Errors for Sample Response Percentages 



Table B1.3 




Overall percentage 
"Acceptable'’ or better 


Standard errars for percentage "Acceptable" or better 
within achievement level ranges 


Basic 

136-176* 


Proficient 

177-214* 


Advanced 
215 and obove* 


1.5 


2.3 


2.1 


— 



* NAEP civics scale range. 

Standard error estimate cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1998 Civics Assessment. 



Table B1.4 I Grade 4: Relationship of the U.S. to other nations 

and to world affairs 



Overall percentoge 
"Correct" 


Standard errors for percentage "Correct" 
within achievement level ranges 


Bosk 

136-176* 


Proficient 

177-214" 


Advanced 
215 and obove* 


1.3 


2.5 


3.4 


— 



* NAEP civics scale range. 

Standard error estimate cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1998 Civics Assessment. 



Table B 1.5 I Grade 4: Roles of U^. citizens in American democracy 



Overall percentage 
"Correct" 


Standard errors for percentage "Correct" 
within achievement level ranges 


Brcic 

136-176* 


Proficient 

177-214" 


Advonced 
215 and obove* 


1.1 


2.2 


3.7 


— 



* NAEP civics scale range. 

Standard error estimate cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1 998 Civics Assessment. 
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Grade 8 




Standard Errors for Sample Response Percentages 
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Table B1.6 I Grade 8: Relationship of the U.S. to other nations 
:$ and to world affairs 





Standard 


errors for percentage "Correct" 




within achievement level ranges 


Overall percentage 


Basic 


Proficient 


Advanced 


"Carrect" 


134-177* 


178-212* 


213 and abave* 


1.1 


1.7 


1.8 


— 



* NAEP civics scale range. 

Standard errar estimate cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment af Educational Progress 
(NAEP), 1998 Civics Assessment. 



Table Bl.^l Grade 8: Foundations of the American political system 





Standard 


errors for percentage "Correct" 




within achievement level ranges 


Overall percentage 


Basic 


Proficient 


Advanced 


“Correct" 


134-177* 


178-212* 


213 and above* 


1.3 


1.9 


3.4 


— 



* NAEP civics scale range, 

Standard error estimate cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1998 Civics Assessment. 



.8 



Grade 8: ^undations of the Anferican political system 





Standard errors for percentage "Complete" 




within achievement level ranges 


Overall percentoge 


Basic 


Proficient 


Advanced 


"Complete" 


134-177* 


178-212* 


213ond above* 


0.7 


1.0 


2.6 


— 



* NAEP civics scale range. 

Standard error estimate cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1998 Civics Assessment. 
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Standard Errors for Sample Response Percentages 
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Table B1.9I Grade 12: Foundations of the 

American political systerry < 



Overall percentage 
"Correct" 


Standard errars far percentage "Carrect" 
within achievement level ranges 


Bosic 

139-173* 


Proficient 

174-203* 


Advanced 
204 and above* 


1.2 


2.1 


2.0 


— 



* NAEP civics scale range. 

Standard error estimate cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEPj, 1998 Civics Assessment. 



Table Bl.iol 


Grade 1 2: Foundations of the American political system 




Overall porcentoge 
' "Complete" 


Standard errars far percentage "Camplete" 
within achievement level ranges 


Bosic 

139-173* 


Proficient 

174-203* 


Advanced 
204 ond above* 


1.3 


1.7 


3.9 


6.4 



* NAEP civics scale range. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1998 Civics Assessment. 



Grade 1 2: Functions and purpose of government 



“Correct" 



1.3 



Standard errars far percentage "Carrect" 
within achievement level ranges 



Bosk 

139-173* 

2.2 



Proficient 

174-203* 

3.7 



Advonced 
204 and above* 

7.1 



* NAEP civics scale range. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1998 Civics Assessment. 
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Standard errors for average civics scale scores by gender: 1998 






Percentage Average 

of students scale score 



Grade 4 



Male 0.9 1 .0 

Female 0.9 0.9 



Grade 8 



Male 0.6 0.9 

Female 0.6 0.8 



Grade 1 2 



Male 0.7 1.1 

Female 0.7 0.8 



SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 




Standard errors for average civics scale scores by race/ 
ethnicity: 1998 





Percentage 
of students 


Average 
scale scare 




Grade 4 1 


White 


0.2 


0.9 


Black 


0.1 


1.2 


Hispanic 


0.2 


1.7 


Asian/Pacific Islander 


0.1 


2.5 


American Indian 


0.2 


3.7 




Grade 8 I 


White 


0.1 


0.9 


Black 


0.1 


1.1 


Hispanic 


0.1 


1.2 


Asian/Pacific Islander 


0.3 


5.8 


American Indian 


0.3 


3.5* 




Grade 1 2 1 


White 


0.4 


0.9 


Black 


0.3 


1.7 


Hispanic 


0.3 


1.3 


Asian/Pacific Islander 


0.2 


4.2 


American Indian 


0.2 


6.3 * 



* Estimate may be unreliable due to small sample size. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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Standard errors for average civics scale scores by parents’ highest level 
of education: 1998 





Percentage 


Average 




of students 


scale score 


i 




Grade 4 1 


Graduated from college 


1.1 


0.9 


Some education after high school 


0.6 


1.4 


Graduated from high school 


0.7 


1.7 


Did not finish high school 


0.6 


3.1 


1 don't know. 


0.5 


2.0 




Grade 8 | 


Graduated from college 


0.9 


0.8 


Some education after high school 


0.5 


1.0 


Graduated from high school 


0.6 


1.2 


Did not finish high school 


0.5 


3.2 


1 don't know. 


0.2 


3.1 




Grade 12 1 


Graduated from college 


1.1 


0.9 


Some education after high school 


0.7 


1.1 


Graduated from high school 


0.5 


1.2 


Did not finish high school 


0.3 


2.1 


1 don't know. 


0.1 


5.3 



SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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Standard errors for average civics scale scores by region: 1998 





Percentage 
of students 


Average 
scole score 




^ Grade, 4 


Northeast 


0.6 


1.5 


Southeast 


0.9 


0.8 


Central 


0.4 


1.7 


West 


1.0 


1.6 




Grcide 8 


Northeast 


0.6 


1.6 


Southeast 


1.0 


1.4 


Central 


0.2 


1.6 


West 


1.2 


1.3 




Grade 12 


Northeast 


1.0 


1.8 


Southeast 


1.0 


1.4 


Central 


0.7 


2.1 


West 


0.9 


0.9 



SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 




Standard errors for average civics scale scores by type of location: 1998 





Percentoge 
of students 


Average 
scale score 




Grade 4 | 


Central city 


1.3 


1.2 


Urban fringe/large town 


1.8 


1.3 


Rural/small town 


1.6 


1.8 




Grade 8 | 


Central city 


1.4 


1.2 


Urban fringe/large town 


1.9 


1.2 


Rural/small town 


1.4 


1.6 




Grade 12 J 


Central city 


1.6 


1.4 


Urban fringe/large town 


1.8 


1.3 


Rural/small town 


1.3 


1.4 



SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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Standard errors for average civics scale scores by Free/Reduced-Price 
School Lunch Program eligibility: 1998 





Percentage 


Average 




of students 


scale score 




-- 


Grade 4 | 


Eligible 


1.1 


0.9 


Not eligible 


1.4 


1.1 


Information not available 


1.5 


2.2 




1 


Grade 8 I 


Eligible 


0.9 


1.1 


Not eligible 


1.7 


1.0 


Information not available 


1.9 


2.2 




1 


Grade 12 I 


Eligible 


0.8 


1.4 


Not eligible 


2.0 


1.0 


Information not available 


2.1 


1.3 




SOURCE: NoHonol Center for EducoHon StofisHcs, 
Notionol Assessment of Educotionol Progress (NAEP), 




1 998 Civics Assessment. 



Table B2.7 




Standard errors for average civics scale scores by type of school: 
1998 





Percentage 
of students 


Average 
scale score 




Public 


0.7 


0.7 


Nonpublic 


0.7 


1.9 


Nonpublic: Catholic 


0.6 


1.7 


Other nonpublic 


0.5 


4.4 




Public 


1.0 


0.7 


Nonpublic 


1.0 


2.8 


Nonpublic: Catholic 


0.8 


1.6 


Other nonpublic 


0.6 


5.9 




Public 


1.0 


0.9 


Nonpublic 


1.0 


1.4 


Nonpublic: Catholic 


0.9 


1.5 


Other nonpublic 


0.4 


3.0 



SOURCE: Nationol Center for Education Statistics, 
NoHonol Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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Standard errors for percentage of students at or above achievement 
levels in civics by gender: 1998 







At or 

Below above At or above 

Basic Basic Proficient Advanced 



Grade 4 1 



Male 

Female 

Male 

Female 




1.1 

1.2 



1.2 

1.0 

1.1 

1.2 



1.2 

1.2 



Grade 8 



1.0 

1.1 



0.4 

0.4 

0.3 

0.3 



Male 
Female I 



1.2 

1.2 



1.2 

1.2 



1.2 

1.1 



0.6 

0.4 



SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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Standard errors for percentage of students within each achievement 
level range in civics by gender: 1998 



;rsEp 

m- 



Below At At At 

Basic Basic Proficient Advanced 



Grg^e 4 



Male 1.2 1.3 1.2 0.4 

Female 1.0 1.0 1.1 0.4 



Grade 8 



Male 


1.1 


1.1 


1.0 


0.3 


Female 


1.2 


1.0 


1.1 


0.3 



■ 


-m- 


Grade 1 2 




Male 


1.2 


1.0 


1.0 


0.6 


Female 


1.2 


1.0 


1.2 


0.4 



SOURCE: National Center for Education Statistics, 
National Assessment of Educotiortal Progress (NAEP), 
1998 Civics Assessment. 
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Standard errors for percentage of students at or above achievement 
levels in civics by race/ethnicity: 1998 



m- 





Below 

Basic 


At or 
above 
Basic 


At or above 
Proficient 


Advanced 




.-5 


Grade 4 


White 


1.1 


1.1 


1.3 


0.5 


Black 


1.8 


1.8 


1.2 


0.3 


Hispanic 


2.2 


2.2 


0.9 


... 


Asian/Pacific Islander 


3.5 


3.5 


4.7 


1.3 


American Indian 


4.0 


4.0 


3.9 


... 




-- 'VI 7 T;?.. 

. * - 


Gjade 8 ^ 


White 


1.0 


1.0 


1.1 


0.3 


Black 


1.7 


1.7 


1.0 


... 


Hispanic 


2.2 


2.2 


0.8 


0.2 


Asian/Pacific Islander 


5.8 


5.8 


4.5 


1.1 


American Indian 


5.7 


5.7 


3.7 


... 






Grade 12 HI 


White 


1.1 


1.1 


1.2 


0.6 


Black 


2.3 


2.3 


1.3 


0.3 


Hispanic 


1.9 


1.9 


1.3 


0.4 


Asian/Pacific Islander i 


3.8 


3.8 


6.2 


2.1 


American Indian 


8.8 


8.8 


4.3 


... 



— Standard error estimate cannot be accurately determined. 

SOURCE: National Center for Education Statistics, 

National Assessment of Educational Progress (NAEP), 

1 998 Civics Assessment. 
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Standard errors for percentage of students within each achievement 
level range in civics by race/ethnicity: 1998 






White 
Black 
Hispanic 
Asian/Pacific Islander 
American Indian 

White 
Black 
Hispanic 
Asian/Pacific Islander 
American Indian 

White 
Black 
Hispanic 
Asian/Pacific Islander 
American Indian 



Below At At At 

Basic Basic Proficient Advanced 




1.1 


1.0 


1.2 


0.5 


1.8 


2.3 


1.2 


0.3 


2.2 


1.9 


0.9 


... 


3.5 


5.8 


4.8 


1.3 


4.0 


4.6 


3.9 


... 




Grade 8 






1.0 


1.0 


1.0 


0.3 


1.7 


1.6 


1.0 


... 


2.2 


2.3 


0.8 


0.2 


.5.8 


4.1 


4.1 


1.1 


5.7 


4.3 


3.8 


— 




1:1 

2.3 

1.9 

3.8 

8.8 



0.8 

2.1 

2.3 

4.0 

8.2 



1.1 

1.2 

1.3 

4.8 

4.1 



0.6 

0.3 

0.4 

2.1 



- - - Standard etror estimate cannot be accurately determined. 

SOURCE: Natianal Center for Education Statistics, 

Natianal Assessment of Educational Progress (NAEP), 

1 998 Civics Assessment. 
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Standard errors for percentage of students at or above achievement 
levels in civics by parents’ highest level of education: 1998 



I risep 



At or 

Below obove At or obove 





Basic 


Bosk 


Proficient 


Advanced 






Grade 4 


Graduated from college 


1.2 


1.2 


1.2 


0.5 


Some education after high school 


2.3 


2.3 


1.9 


0.6 


Graduated from high school 


2.8 


2.8 


2.9 


... 


Did not finish high school 


5.3 


5.3 


3.2 


... 


1 don't know. 


2.6 


2.6 


2.7 


... 






Grade 8 


Graduated from college 


1.0 


1.0 


1.2 


0.4 


Some education after high school 


1.5 


1.5 


1.2 


0.2 


Graduated from high school 


2.0 


2.0 


1.4 


0.2 


Did not finish high school 


3.8 


3.8 


1.2 


... 


1 don't know. 


4.1 


4.1 


2.2 


... 






Grade 12 


Graduated from college 


1.0 


1.0 


1.3 


0.8 


Some education after high school 


1.3 


1.3 


1.4 


0.4 


Graduated from high school 


1.8 


1.8 


1.4 


0.2 


Did not finish high school 


2.6 


2.6 


1.6 


... 


1 don't know. 


6.7 


6.7 


... 


... 



• - - Standard error estimate canrwt be accurately determined. 

SOURCE: National Center for Education Statistics, 

National Assessment of Educational Progress (NAEP), 

1 998 Civics Assessment. 
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Standard errors for percentage of students within each achievement 
level range in civics by parents’ highest level of education: 1998 






Below 

Basic 


At At 

Basic Proficient 


At 

Advanced 


1 Grade 4 | 



Graduated from college 


1.2 


0.9 


1.2 


0.5 


Some education after high school 


2.3 


2.2 


2.0 


0.6 


Graduated from high school 


2.8 


3.3 


2.8 


... 


Did not finish high school 


5.3 


6.4 


2.2 


... 


1 don't know. 


2.6 


2.8 


2.8 


... 




Grade 8 


Graduated from college 


1.0 


1.1 


1.2 


0.4 


Some education after high school 


1.5 


1.6 


1.2 


0.2 


Graduated from high school 


2.0 


2.0 


1.4 


0.2 


Did not finish high school 


3.8 


4.0 


1.2 


... 


1 don't know. 


4.1 


4.2 


2.2 


... 




Grade 12 


Graduated from college 


1.0 


1.2 


1.0 


0.8 


Some education after high school 


1.3 


1.4 


1.3 


0.4 


Graduated from high school 


1.8 


2.3 


1.3 


0.2 


Did not finish high school 


2.6 


2.2 


1.6 


... 


1 don't know. 

1 


6.7 


6.9 


... 


... 



• • • Standard error estimate cannot be accurately determined. 

SOURCE: National Center for Education Stotistics, 

National Assessment of Educational Progress (NAEP), 

1 998 Civics Assessment. 
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Standard errors for percentage of students at or above achievement 
levels in civics by region: 1998 



m- 








Below 


At or 








above At or above 






Basic 


Basic 


Proficient 


Advanced 






Grad%4 | 


Northeast 


1.9 


1.9 


1.9 


0.7 


Southeast 


1.3 


1.3 


1.5 


0.4 


Central 


2.4 


2.4 


2.2 


0.7 


West 


2.0 


2.0 


1.5 


0.3 




i 


Grade 8 


Northeast 


1.9 


1.9 


2.1 


0.4 


Southeast 


1.8 


1.8 


1.5 


0.4 


Central 


1.7 


1.7 


1.3 


0.6 


West 


1.6 


1.6 


1.3 


0.3 






Grade 12 1 


Northeast 


2.3 


2.3 


1.6 


0.5 


Southeast 


2.1 


2.1 


1.1 


0.6 


Central 


2.3 


2.3 


3.2 


1.1 


West 


1.0 


1.0 


1.3 


0.5 



SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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Standard errors for percentage of students within each achievement 
level range in civics by region: 1998 



m. 






Below 

Bosk 


At 

Basic 


At 

Proficient 


At 

Advanced 


] 


. ■:>; 


Grade 4 


1 


Northeast 


1.9 


1.7 


1.9 


0.7 


Southeast 


1.3 


1.5 


1.3 


0.4 


Central 


2.4 


1.8 


2.2 


0.7 


West 


2.0 


1.9 


1.5 


0.3 






Grade 8 




Northeast 


1.9 


1.6 


2.2 


0.4 


Southeast 


1.8 


1.6 


1.4 


0.4 


Central 


1.7 


1.4 


1.4 


0.6 


West 


1.6 


1.1 


1.3 


0.3 






Grade 12 




Northeast 


2.3 


1.4 


1.6 


0.5 


Southeast 


2.1 


1.6 


1.1 


0.6 


Central 


2.3 


2.1 


2.6 


1.1 


West 


1.0 


1.1 


1.1 


0.5 



SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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Standard errors for percentage of students at or above achievement 
levels in civics by type of location: 1998 






Below 

Basic 


At or 

above At or above 
Basic Proficient 


Advanced 






Gracte 4 






Central city 


1.2 


1.2 


1.5 


0.4 


Urban fringe/large town 


2.0 


2.0 


1.3 


0.4 


Rural/small town 


2.1 


2.1 


2.2 


0.7 






Grades 


Central city 


1.6 


1.6 


1.3 


0.3 


Urban fringe/large town 


1.4 


1.4 


1.5 


0.4 


Rural/small town 1 


1.7 


1.7 


1.5 


0.5 






Grade 1 2 


Central city 


1.6 


r;; 


1.6 


0.7 


Urban fringe/large town 


1.6 


1.6 


1.5 


0.5 


Rural/small town 


2.0 


2.0 


1.8 


0.7 



SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1998 Civics Assessment. 
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Figure B3. 5 
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Standard errors for percentage of students within each achievement 
level range in civics by type of location: 1998 








Below 


At 


At 


At 




Bosic 


Basic 


Proficient 


Advanced 






Grade 4 




Central city 


1.2 


1.3 


1.3 


0.4 


Urban fringe/large town 


2.0 


1.6 


1.3 


0.4 


Rural/small town 


2.1 


1.6 


2.3 


0.7 






Grade 8 




Central city 


1.6 


1.3 


1.3 


0.3 


Urban fringe/large town 


1.4 


1.2 


1.5 


0.4 


Rural/small town 


1.7 


1.2 


1.5 


0.5 






Grade 12 




Central city 


1.6 


1.3 


1.3 


0.7 


Urban fringe/large town 


1.6 


1.3 


1.4 


0.5 


Rural/small town 


2.0 


1.7 


1.5 


0.7 



SOURCE: Notional Center for Education Statistics, 
Notionol Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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Standard errors for percentage of students at or above achievement 
levels in civics by Free/Reduced-Price School Lunch Program 
eligibility: 1998 



m 





Below 

Basic 


At or 

above At or above 
Basic Proficient 


Advonced 




Grade 4 


Eligible 


1.3 


1.3 


0.9 




Not eligible 


1.4 


1.4 


1.3 


0.5 


Information not available i 


3.1 


3.1 


2.5 


0.9 






Grade 8 


Eligible 


1.6 


1.6 


0.8 


0.2 


Not eligible 


1.2 


1.2 


1.2 


0.3 


Information not available 


2.7 


2.7 


2.1 


0.6 






Grade 12 


Eligible 


2.1 


2.1 


1.7 


0.4 


Not eligible 


1.1 


1.1 


1.3 


0.5 


Information not available 


1.6 


1.6 


1.5 


0.7 



— Standard error estimate cannot be accurately determined. 

SOURCE: National Center for Education Statistics, 

Notional Assessment of Educational Progress (NAEP), 

] 998 Civics Assessment. 
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Figure B3.6 




the MAHON’S 



REPORT 

CARD 



roaep 

m: 



Standard errors for percentage of students within each achievement 
level range in civics by Free/Reduced-Price School Lunch Program 
eligibility: 1998 




Eligible 
Not eligible 
Information not available 

Eligible 
Not eligible 
Information not available 

Eligible 
Not eligible 
Information not available 



Below 

Basic 


At 

Basic 


At 

Proficient 


At 

Advanced 




Grade 4 


- 


1.3 


1.4 


1.0 


... 


1.4 


1.3 


1.3 


0.5 


3.1 


2.9 


2.0 


0.9 


1 Grade 8 I 


1.6 


1.5 


0.8 


0.2 


1.2 


0.9 


1.2 


0.3 


2.7 


2.0 


2.1 


0.6 




2.1 


1.8 


1.5 


0.4 


1.1 


1.0 


1.1 


0.5 


1.6 


1.2 


1.5 


0.7 



- - - Standord error estimate connot be occurotely determined. 

SOURCE: Notionol Center for Educotion StatisHcs, 

Notionol Assessment of Educotionol Progress (NAEP), 

1 998 Civics Assessment. 
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Standard errors for percentage of students at or above achievement 
levels in civics by type of school: 1998 











At or 








Below 


above 


At or above 






Basic 


Basic 


Proficient 


Advanced 






Grade 4 




Public 


1.0 


1.0 


0.9 


0.4 


Nonpublk 


2.5 


2.5 


2.4 


0.9 


Nonpublic: Catholic 


1.8 


1.8 


2.8 


1.1 


Other nonpublic 


6.0 


6.0 


5.0 


1.7 






Grade 8 




Public 


1.0 


1.0 


0.8 


0.2 


Nonpublic 


2.8 


2.8 


2.4 


0.7 


Nonpublic: Catholic 


1.5 


1.5 


3.0 


0.8 


Other nonpublic 


5.9 


5.9 


4.3 


1.4 


1 




Grade 12 


■ 


Public 


1.0 


1.0 


1.0 


0.4 


Nonpublic 


1.8 


1.8 


2.1 


1.2 


Nonpublic: Catholic 


2.0 


2.0 


2.4 


1.3 


Other nonpublic 


3.9 


3.9 


3.8 


2.7 



SOURCE: Notionol Center for Educotion Statistics^ 
Notionol Assessment of Educotiono) Progress (NAEP), 
1 998 Civics Assessment. 
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Figure B3.7 



THE NAnON'S 
REPORT 



Standard errors for percentage of students within each achievement 
level range in civics by type of school: 1998 



m- 






Below 


At 


At 


At 




Basic 


Basic Proficient 


Advanced 




Gracie 4 J 


Public 


1.0 


0.9 


1.0 


0.4 


Nonpublic 


2.5 


2.5 


2.2 


0.9 


Nonpublic: Catholic 


1.8 


2.7 


2.5 


1.1 


Other nonpublic 


6.0 


5.0 


4.5 


1.7 






Grade 8 




VI j 


Public 


1.0 


0.7 


0.9 


0.2 


Nonpublic 


2.8 


2.8 


2.4 


0.7 


Nonpublic: Catholic 


1.5 


2.4 


2.9 


0.8 


Other nonpublic 


5.9 


5.2 


4.0 


1.4 




Grade 12 | 


Public 


1.0 


0.8 


0.9 


0.4 


Nonpublic 


1.8 


1.8 


1.7 


1.2 


Nonpublic: Catholic 


2.0 


1.7 


2.1 


1.3 


Other nonpublic 


3.9 


3.6 


2.9 


2.7 



SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1998 Civics Assessment. 
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Table B4.1 




Standard errors for percentage of students, average civics scale 
scores, and percentage at or above Proficient by teacher’s highest 
degree, grades 4 and 8: 1998 



THE HAnOirS 
REPORT 
CARO 



m 



What is the %6esf aaal^au 



degree you Md? 


Grade 4 


Grade 8 


Kgb Sd^ Assodfltf's 
or Vocfitioaid Coilffbiftioii 

Percentage of students 
Average scale score 
Percentage at or above Proficient 


0.2 


0.2 


Bachelor's Degree 






Percentage of students 


2.3 


1.9 


Average scale score 


1.1 


0.9 


Percentage at or above Proficient 


1.5 


1.1 


Mast^s Degree 






Percentage of students 


2.0 


1.9 


Average scale score 


1.0 


1.2 


Percentage at or above Proficient 


1.3 


1.4 


Edocotion SpedoDst Degree 






Percentage of students 


0.8 


0.8 


Average scale score 


5.1 


2.9 


Percentage at or above Proficient 


6.8 


3.2 


Doctorate or Professionoi Degree 






Percentage of students 


0.4 


0.4 


Average scale score 


— 


6.4 


Percentage at or above Proficient 


— 


6.9 



Standard error estimate canriot be accurately determined. 

SOURCE: National Center for Educotian Statistics, Nationol Assessment of Educotianol Progress (NAEP), 1 998 
Civics Assessment. 
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Table B4.2 




Standard errors for percentage of students, average civics scale 
scores, and percentage at or above Proficient by teacher’s 
undergraduate major, grades 4 and 8: 1998 



THE m\w$ 
REPORT 
CARD 



m 



What was your umhrgraduate major? 


1 Grade 4 

■ X- 


Grade 8 


History 


1 

i 




Percentage of students 


0.7 


, 1.7 


Average scale score 


4.7 


1.0 


Percentage at or above Proficient 


4.3 


1.2 


Political Sdeoce 






Percentage of students 


0.4 


1.2 


Average scale score 


— 


2.8 


Percentage at or above Proficient 


— 


4.2 


Education 






Percentage of students 


1.3 


2.2 


Average scale score 


0.8 


1.5 


Percentage at or above Proficient 


1.2 


1.9 


Other 






Percentage of students 


1.1 


1.5 


Average scale score 


2.2 


1.5 


Percentage at or above Proficient 


2.2 


1.6 



Standard error estimate cannot be accurately determined. 

SOURCE: Notional Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1 998 
Civics Assessment. 
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Standard errors for percentages of students, average civics scale scores, 
and percentage at or above Proficient by teacher’s type of certification, 
grades 4 and 8: 1998 



Type of feaddttg tertifkatm Md bi 


Percentage 
of students 


Average 
scale score 


Percentage at or 
above Profident 


mdn as^maeat mm 


Grade 4 ^ | 


Advanced Professional 


1.4 


2.1 


3.0 


Regular 


1.6 


0.8 


1.1 


Probationary 


0.5 


4.0 


4.9 


Temp/Provisional 


0.7 


4.0 


3.0 


Other 


0.2 


— 


— 


Don't have 


0.4 


7.0 


6.1 




Grade 8 I 


Advanced Professional 


1.9 


2.2 


2.8 


Regular 


2.1 


0.9 


1.0 


Probationary 


• 0.5 


3.3 


3.2 


Temp/Provisional 


0.6 


3.9 


4.4 


Other 


0.2 


5.2 


7.4 


Don't have 


0.6 


10.7 


6.2 



Standard error estimate cannot be accurately determined. 

SOURCE: National Center for Education Statistics, 

National Assessment of Educational Progress (NAEP), 

1 998 Civics Assessment. 
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Table B4.4 






Standard errors for percentages of students, average civics scale 
scores, and percentage at or above Proficient by years of general 
teaching experience, grades 4 and 8: 1998 



the MOTON'S 
RBHWT 
CMU) 






Yean of ^meatary or seamhry 


Percentage 
of students 


Average 
scale score 


Percentage at or 
above Proficient 


teat^ experkare | 




Grade|^ 


^ _l 


2 years or less 


1.1 


2.3 


2.9 


3-5 years 


1.0 


2;i 


2.8 


6-1 0 years 


1.0 


1.5 


2.0 


1 1 -24 years 


1.5 


1.2 


1.3 


25 years or more 


1.6 


1.3 


2.2 




2 years or less 


1.0 


2.4 


2.3 


3-5 yeors 


1.5 


2.1 


2.2 


6-1 0 years 


1.4 


1.8 


2.1 


1 1 -24 years 


1.8 


1.3 


1.7 


25 years or more 


1.8 


1.8 


1.7 



SOURCE: National Center for Education Statistics, 
Notional Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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RBHmr 



Standard errors for percentage of students, average civics scale scores, 
and percentage at or above Proficient by years teaching government/ 
civics, grades 4 and 8: 1998 






Yem of feaddng go¥&meat or dvks 


Percentage 
of students 


Average 
scale score 


Percentage at or 
above Proficient 


1 


Grade 4 


* 


None 


— 


— 


— 


2 years or less 


2.5 


1.6 


1.9 


3-5 years 


1.4 


2.4 


3.0 


6-10 years 


1.2 


2.3 


3.2 


1 1-24 years 


2.0 


1.8 


2.4 


25 or more years 


0.9 


2.4 


3.1 






^ Grade 8 




None 


2.6 


1.6 


2.0 


2 years or less 


2.1 


2.0 


2.0 


3-5 years 


1.7 


2.9 


3.8 


6-10 years 


2.4 


3.0 


2.9 


1 1-24 years 


1.7 


2.5 


2.6 


25 or more years 


1.2 


3.2 


3.3 



Standard error esHmate cannot be accurately determined. 

SOURCE: National Center for Education Statistics, 

National Assessment of Educational Progress (NAEP), 

1 998 Civics Assessment. 
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Standard errors for percentage of students, average civics scale scores, 
and percentage at or above Proficient by amount of time teachers 
reported spending in professional development workshops in social 
studies during the last twelve months, grades 4 and 8: 1998 



Vme soeat ia pntesshal Avdopmait 
workshops ia sodd storks ia the past , 

twelve aumtbs 


Percentage 
of students 


Average 
scale score 


Percentage at or 
above Proficient 




Grade 4 




None 


2.0 


1.3 


1.7 


Less than 6 hours 


2.1 


1.5 


1.8 


6-15 hours 


1.1 


2.3 


3.2 


16-35 hours 


0.8 


3.5 


3.6 


More than 35 hours 


0.5 


4.4 


5.1 




None 


2.1 


1.6 


1.5 


Less than 6 hours 


1.8 


1.6 


1.5 


6-15 hours 


2.1 


1.5 


1.5 


16-35 hours 


1.4 


1.8 


2.6 


More than -35 hours 


1.5 


2.5 


3.8 



SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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Table B4.7 




Standard errors for percentage of students, average civics scale 
scores, and percentage at or above Proficient by teacher’s 
reported preparation, grades 4 and 8: 1998 



THE NCTOM’S 
REPORT 



m- 



Grade 4 I Grade 8 



How wbI pnpond 
are yookfbe folowbg? 


Percentage 

of 

students 


Average 

scale 

score 


Percentage 
at or above 
Proficient 


Percentoge 

of 

students 


Average 

scale 

score 


Percentage 
at or obove 
Proficient 


Sodd stMfies mstroction 














Well prepared 


2.0 


1.1 


1.4 


1.7 


0.9 


1.1 


Moderately well prepared 


1.8 


1.1 


1.4 


1.6 


1.4 


1.9 


Not well prepared 


0.7 


3.8 


3.4 


0.5 


5.9 


4.7 


Using Instmctioiml moteriols in social stmfes 














Well prepared 


1.8 


1.2 


1.5 


2.6 


1.0 


1.3 


Moderately well prepared 


1.6 


1.2 


1.5 


2.5 


1.0 


1.2 


Not well prepared 


1.0 


2.8 


2.3 


0.8 


4.1 


4.2 


Qas%iom (fimote and govOTumco 














Well prepared 


1.6 


1.1 


1.3 


1.9 


0.8 


1.2 


Moderately well prepared 


1.5 


1.3 


1.8 


1.7 


1.5 


1.6 


Not well prepared 


0.6 


2.3 


3.0 


0.6 


3.6 


4.2 


Using vofamtary notional stondank for civics 














Well prepared 


0.8 


3.4 


4.5 


1.3 


2.3 


2.2 


Moderately well prepared 


1.4 


1.7 


2.1 


2.5 


1.3 


1.4 


Not well prepared 


1.5 


0.9 


1.2 


2.5 


1.2 


1.3 


Using softwore for sodol studies 














Well prepared 


1.0 


3.7 


4.2 


1.4 


2.3 


2.7 


Moderately well prepared 


1.5 


1.4 


2.1 


2.1 ■ 


1.0 


1.1 


Not well prepared 


1.7 


0.9 


1.2 


2.2 


1.4 


1.6 



SOURCE: National Center for Education Statistics, Notional Assessment of Educational 
Progress (NAEP), 1998 Civics Assessment. 
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Standard errors for percentage of students, average civics scale 
scores, and percentage at or above Proficient by teacher’s 
reported preparedness to fulfill certain teaching- related tasks, 
grades 4 and 8: 1998 



1 




Grade 8 | 


1 

How wei prepared 
are you m the Mhwkig: 


Percentage 

of 

students 


Average 

scale 

score 


Percentage 
at or above 
Proficient 


Percentage 

of 

students 


Average 

scale 

score 


Percentage 
at ar abave 
Prafident 


r 

Use of telecoramonicorions 














Well prepared 


1.3 


2.0 


2.7 


1.7 


1.8 


1.6 


Moderately well prepared 


1.8 


1.0 


1.7 


2.2 


0.9 


1.3 


Not well prepared 


2.1 


1.2 


1.6 


2.0 


1.8 


1.9 


Use of compoters 














Well prepared 


1.8 


1.6 


2.7 


1.9 


1.6 


1.5 


Moderately well prepared 


1.5 


0.8 


1.0 


2.1 


1.0 


1.2 


Not well prepared 


1.4 


2.0 


2.4 


1.3 


2.6 


2.7 


Cooperative |roop iastnictioa 














Well prepared 


1.5 


1.0 


1.5 


2.2 


1.1 


1.3 


Moderately well prepared 


1.5 


1.2 


1.4 


2.0 


1.3 


1.4 


Not well prepared 


0.5 


5.4 


5.4 


0.9 


4.0 


3.3 


Oflssroom monogefflent ond orgonizotion 


i 












Well prepared 


1.3 


0.8 


1.2 


1.5 


0.8 


1.1 


Moderately well prepared 


1.3 


2.0 


2.2 


1.4 


2.0 


2.2 


Not well prepared 


0.2 


— 


— 


0.3 


6.3 


7.3 



Standard error estimate cannot be accurately determined. 

SOURCE; Nafional Center for Educafion StaHsHcs, NaHonal Assessment of Educational 
Progress (NAEP), 1998 Civics Assessment. 
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Table B4.9 




THE NAnON'S 



REPORT t dPti 
CARD 






Standard errors for percentage of students, average civics scale scores, 
and percentage at or above Proficient by class time per day spent on 
socid studies instruction, grades 4 and 8: 1998 



How moth time do you spend with 
this doss hi sodd stmBes 



hstnuHoB m a tj^dcd dayj 


Grade 4 


Grade 8 


Less thon 30 mfoites 






Percentage of students 


1.0 


0.9 


Average scale score 


2.0 


8.9 


Percentage at or obove Proficient 


2.3 


6.6 


30-44 minutes 






Percentage of students 


1.8 


2.5 


Average scale score 


1.0 


1.2 


Percentage at or above Proficient 


1.3 


1.5 


45-60 minutes 






Percentage of students 


1.9 


2.5 


Average scale score 


2.0 


1.2 


Percentage at or above Proficient 


2.2 


1.3 


Greoter than 60 mnstes 






Percentage of students 


0.3 


0.9 


Average scale score 


— 


2.2 


Percentage at or above Proficient 


— 


2.3 



Standard error estimate cannot be accurately determined. 

SOURCE: Notional Center for Education Statistics, Notional Assessment of Educational 
Progress (NAEP), 1 998 Civics Assessment. 
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Table B4.10 



Standard errors for percentage of students, average civics 
scale scores, and percentage at or above Proficient by teachers’ 
reports on the availability of resources, grades 4 and 8: 1998 



THE wmows 
REPORT 



m 



WU(b b tme about how well your 
sthool systm provides yoo with the 
bisttmtioMd materUds and other 



resomes yoo med to teadi your doss? 


Grade 4 


Grade 8 


Ail 

Percentage of students 
Average scale score 
Percentage at or above Proficient 


1.5 

1.7 

2.2 


1.6 

2.0 

2.3 


Most 

Percentage of students 
Average scale score 
Percentage at or above Proficient 


1.6 

1.1 

1.4 


1.6 

1.1 

1.2 


Some 

Percentage of students 
Average scale score 
Percentage at or above Proficient 


1.5 

1.2 

1.4 


2.1 

1.3 

1.3 


None Q 2 

Percentage of students 
Average scale score 


0.2 



Standard eiTOf estimate cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP), 1 998 Civics Assessment. 
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Table Bill 



THE NAnON'S 
REPORT 



m. 



Standard errors for percentage of students, average civics scale scores, 
and percentage at or above Proficient by availability of computers, 
grades 4, 8, and 12: 1998 



An avaSaUe to 

stod&ih b yoar doaes m aoy 
ol t^ fdowtag ways? 




YESdtt 






NOi 




Percentage 
of students 


Average 
scale score 


Percentage at or 
oboye Proficient 


Percentoge 
of students 


Average 
scale score 


Percentoge ot or 
above Proficient 


Grmfe4 














Available in all classrooms 


1.8 


0.9 


1.1 


1.8 


1.9 


2.2 


Grouped in a separate 
computer lab available 
to classes 


2.2 


0.9 


1.0 


2.2 


2.4 


2.3 


Available to bring to 
classrooms when needed 


2.8 


1.5 


1.7 


2.8 


1.2 


1.4 


Grade 8 














Available in all classrooms 


3.0 


1.2 


1.3 


3.0 


1.5 


1.4 


Grouped in a separate 
computer lab available 
to classes 


1.7 


0.8 


0.9 


1.7 


3.0 


1.9 


Available to bring to 
classrooms when needed 


2.2 


1.3 


1.3 


2.2 


1.0 


1.0 


Gfwie 12 

Avaibble in all classrooms 


2.2 


1.9 


2.5 


2.2 


1.0 


1.2 


Grouped in a separate 
computer lab available 
to classes 


1.3 


0.9 


1.1 


1.3 


3.7 


3.9 


Available to bring to 
classrooms when needed 


3.1 


1.5 


1.9 


3.1 


1.2 


1.3 



SOURCE: Notiono! Center for Educotion Statistics, Notionol Assessment of Educotionol Progress (NAEP), 1 998 Civics Assessment. 
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Table B4.12 



Standard errors for percentage of students, average civics scale 
scores, and percentage at or above Proficient by use of computers 
and the Internet for social studies instruction, as reported by 
teachers, grades 4 and 8: 1998 



Grade 4 



THE M AnON’S 
EPORT 
CARD 



m: 





Percentage 


Average 


Percentage ot or 


. Percentoge 


Average 


Percentage at or 


1 


of students 


scole score 


above Proficient 


of students 


scole score 


above Proficient 


J 

AviddiSty of computers fai 
sodid stones doss 














Nat available 


1.3 


2.3 


2.6 


2.3 


3.1 


2.3 


Lab/librory but difficult ta access 


1.3 


1.6 


2.0 


2.3 


1.3 


1.6 


Readily accessed in lab/library 


1.5 


2.5 


3.3 


2.3 


1.6 


1.8 


Available in the classraam 


2.3 


1.2 


1.3 


1.5 


2.1 


2.3 


Use compnter software 
Every day 


0.5 






0.4 


6.3 


7.3 


Once ar tvirice a week 


0.8 


2.7 


3.6 


1.1 


2.5 


3.3 


Once ar twice a month 


1.4 


1.7 


2.1 


1.9 


1.3 


1.3 


Never or hardly ever 


1.5 


0.9 


1.2 


2.3 


1.3 


1.4 


Access to farfonnolion through 
faitarat in Nn dosvooD 














Everyday 


0.3 


— 


— 


0.3 


— 


— 


Once or twice a week 


0.9 


5.0 


5.5 


1.2 


2.3 


2.6 


Once or twice a month 


• 1.7 


2.2 


2.6 


2.7 


1.4 


1.5 


Never or hardly ever 


2.0 


0.8 


1.2 


2.5 


1.3 


1.4 



Standarxi error estimate connot be occurately determined. 

SOURCE: Notionol Center for Education Statistics, Notionol Assessment of Educotionol Progress (NAEP), 1 998 Civics Assessment. 
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Table B5.1 



THE NAnON’S 
REPORT 



Standard errors for percentage of students, average civics scale scores, 
and percentage at or above Proficient by type of content studied this 
year as reported by students, grade 4: 1998 



m-. 






Hds sdtool yev, 
hewe you stmM any 
the foBowing to/Ba? 


Percentage 
of students 


Yes 

Averoge 
scale score 


Percentage 
at or above 
Profident 


Percentage 
of students 


No 

Average 
scale score 


Percentoge 
at or obavc 
Profident 


1 Don't know 

! Percentage Average 
of students scole score 


Percentoge 
at or obove 
Proficient 


How our 

govemmont worb 


1.2 


0.9 


1.4 


1.0 


1.2 


2.3 


0.7 


1.0 


1,1 


Rules/lows of government 


1.2 


0.8 


1.1 


1.1 


1.3 


2.2 


0.6 


1.5 


1,9 


flections ond voting 


1.2 


0.8 


1.5 


1.3 


1.3 


2.1 


0.6 


1.6 


1,6 


The President/ 




















leaders of country 


1.1 


0.9 


1.2 


1.0 


1.4 


2.1 


0.7 


1.3 


1,7 


Your community 


0.8 


0.7 


1.0 


0.6 


1.7 


2.7 


0.7 


1.4 


1,6 


Rights ond 




















responsibilities of citizens 


1.2 


0.9 


1.3 


1.0 


1.3 


2.1 


0.7 


1.2 


1,4 


How people 




















solve (fisogreements 


1.1 


0.9 


1.3 


0.8 


1.2 


2.4 


0.8 


1.4 


1,9 



SOURCE: NaHonal Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1998 Civics Assessment. 
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~SfancJard error esHmate cannot be accurately determined. 
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Table B5.5 



... 



THE wno»’s 
REPORT 
CARD 



Summary of standard errors of paired percentages of students and 
paired scale score means with significant differences between less 
experienced (two years or less) and experienced (three years or more) 
teachers by selected types of instructional activities, grade 4: 1998 



m- 







frequency of instructional activity 




1 


How often ih you use tbe hBowitg 
resounes to feuA sofkJ sfutfiss bi 
tUs dass? 


Every doy 


Once or twice 
a week 


Once w twke 
amonth 


Never or 
hardly eva 


Perceili^e 
of stedoots 


/brofogo 
safe son 


Perantoge 

ofstodeirts 


Average 
scale score 


Perceetogo 
of studeBls 


safe score 


Percoitt^ 
of stod^ 


fbtenip 
scale score 


Social studies textbook 


] 


27-0.9 




4.2-1 .6 




S.9-4.4 






Books, newspapers, magazines 


i 

1 






2.3-1 .5 




2.7-1 .2 




5.6-^.9 


Primary documents 








4.9-2.7 




3.2-17 




3.3-1. 1 


Quantitative data, charts, or graphs 


i 


3.7-2. 1 


4.2-1. B 


3.1-1.1 




3.0-2.4 






Camputer software 


3 

4 

j 








3.1-l.S 




1 

1 


1.8-1 .3 


Films, videos, filmstrips 




1.9-1 .6 






2.5-1.1 


i 

j 

4 




Have students complete a worksheet 


j 

j 






2.9-1 .0 




3.8-2.0 


1 

4 


6.6-3.S 


Give a lecture 


i 

1 


4.1-2.1 




3.3-1 .2 






19-1.7 




Have students da a group activity/project 
Hove students write a three or 


1 

] 






i 

3.6-17 




2.8-1. 1 




6.4-2.3 


more page report 


1 

i 

> 

1 

j 

i 

i 














2.1-1.2 


Have students participate in debates 










2.8-2.1 


3 

5 4.4-2.0 


3.1-0.9 


Have students participate in mock trials 










3.3-1 .9 




1^ 

1. 9-1.1 


Have students write letters 


S 

i 










2.5-1 .6 


i 


2.1-1 .4 



Indicates a significant difference in favor of experieiKed teochers. 

SOURCE: National Center for Education Statistics, National Assessment of Educotional Progress (NAEP), 1998 Civics Assessment. 
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Table 85.6 ;^ 



THE WATKW^S 
REPORT (j 
CARO 



Summary of standard errors for examples of pairs of percentages of 
students with significant differences between social studies teachers 
who participated in social studies workshops in the past year and those 
who did not, by type of instructional activity, grades 4 and 8: 1998 



Grade 8 



resomes to teaib sodid stwBes in | 

Hds dttss? 1 


'1^' 

sore .s 


Wot ’ TOOT 


Social sti^ textbmik 








»> 

4.2-18 












Books, newspopers, nmgozittes 


0.7-1 .4 




4.4-17 




Primary doaiments 


2.0-16 




0.4-1 .0 




Quantitative date, charts, or graphs 










Computer sofMnire 


1.5-12 




18-2.3 




Rims, videos, filmstrips 










Have students complete a worksheet 




1.1-1 .4 






Give a lecture 










Have students do a group ocHvity/project 




11-1.3 






Have students write a three ar 










more poge report 






4.1-16 




Hove students portidpote in debotes 


11-2.4 




17-2.6 




Have students partidpote in mock triak 


15-12 




4.4-2.9 




Have stutfents write letters 


10-1.9 




2.5-15 





^ Indicates o significant difference in the percentage of students experiencing the instructionol activity. 

SOURCE: Notionol Center for Education Statistics, Notional Assessment of Educotionol Progress (NAEP), 
1 998 Civics Assessment. 
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THE liATION’S 
REPORT 
CARD 



Standard errors for percentage of students, average civics scale scores, 
and percentage at or above Proficient by frequency of discussion of 
school studies at home, grades 4, 8, and 12: 1998 






Now ahea do you disaus things 
yea haae feoniei/ at sdiod wM 
smeone at home? 



Pmentage Averoge Percentage at or 

ol students scale score above Proficient 




Daily 


1.0 


0.9 


1.3 


Once or twice a week 


0.7 


1.3 


1.9 


Once or twice a month 


0.4 


2.1 


2.9 


Never or hardly ever 


0.6 


1.3 


1.5 






Grade 8 




Daily 


0.7 


0.8 


1.1 


Once or twice a week 


0.6 


1.3 


1.5 


Once or twice a month 


0.4 


1.7 


2.2 


Never or hardly ever 


0.6 


1.0 


1.1 




Grade 12 1 


Daily 


0.6 


0.9 


1.3 


Once or twice a week 


0.6 


1.0 


1.8 


Once or twice a month 


0.4 


1.5 


1.9 


Never or hardly ever 


0.5 


1.0 


1.2 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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Table B5.8 




THE HATION’S 
REPORT 






Standard errors for percentage of students, average civics scale scores, 
and percentage at or above Proficient by volunteer work status, 
grade 12: 1998 



Did foo do Yokmteer work kj 
ywff coammDy fhb year? 


■ 


Grade 12 


; m 


Percentage 
of students 


Average 
scale score 


Percentage at or 
above Proficient 


Yes, with my school 


0.8 


1.0 


1.4 


Yes, on my own 


0.8 


1.0 


1.7 


No 


0.9 


0.9 


0.9 



SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 






Table B5.9 



THE NAnON’S 
REPORT 
CARO 



m- 



Standard errors for percentage of students, average civics scale scores, 
and percentage at or above Proficient by hours per week 
wortog at a job for pay, grade 12: 1998 



How many bom do you 
work of a job tor pay? 




Grade 1 2 


1 


Percentage 
of students 


Average 
scale score 


Percentage at or 
above Proficient 


None 


0.9 


0.9 


1.2 


1-5 hours 


0.3 


2.4 


3.1 


6-10 hours 


0.4 


1.7 


2.8 


11-15 hours 


0.4 


1.5 


2.7 


1 6-20 hours 


0.5 


1.3 


1.7 


2 1 or more hours 


0.6 


1.1 


1.2 



SOURCE: National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 
1 998 Civics Assessment. 
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